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ABSTRACT 

Data from the 1990 National Assessment of Educational 
Progress (NAEP) and the National Education Longitudinal Study of 1988 
(NELS) are analyzed for eighth graders to distinguish the 
charact er ist ics of low-achieving groups and the independent 
predictors of low test scores. Results provide a basis for evaluating 
the adequacy of the NAEP for these purposes. Analyses compared the 
bottom decile and quartile on the test-score distribution to the 
eighth-grade population as a whole in terms of simple univariate 
statistics. Logistic regression analysis was used to estimate the 
independent relat i onships between an individual's low achievement 
levels and social context character ist ics in the NELS. Finally, 
several ordinary regressions and bivariate correlations among the 
social context measures were themselves analyzed to assess the 
adequacy of proxies in the NAEP. Findings demonstrate that several of 
the characterist ics presently in the NAEP are useful for 
differentiating low achievers as a group from the eighth grade 
population as a whole. Analyses of the NELS show, however, that the 
NAEP currently lacks several measures important for the purpose. 

These include low family income, low levels of family closure, large 
families, single parent households, mothers who were young when they 
gave birth, greater school mobility, grade retention, lower grade 
point averages, low school mean income, and low levels of closure in 
the school as a whole. (Contains 12 tables, 17 figures, and 185 
references.) (SLD) 
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This report is intended for educators, researchers, and policymakers concerned 
with the adequacy of the NAEP for monitoring the achievement of students at risk of 
educational failure (i.e., low test scores). It analyzes the 1990 NAEP data for eighth 
graders and the eighth grade National Education Longitudinal Study (1988). Analyses 
of these data provide useful information for examining the distinguishing 
characteristics of low-achieving groups and the independent predictors of low test 
scores, and they provide a basis for evaluating the adequacy of NAEP for this purpose. 

This project was conducted by RAND as part of the work of the NAEP Technical 
Review Panel (TRP) under contract to the National Center for Education Statistics, U.S. 
Department of Education. The TRP is a joint endeavor of the University of California, 
Los Angeles; RAND; and the University of Colorado at Boulder. The opinions 
presented in this report, however, are solely those of the authors and do not represent 
the position of RAND or the National Center for Education Statistics. 



SUMMARY 



This study is intended to assess the adequacy of the NAEP for describing 
students at risk of educational failure in terms of the social contexts in which they live 
(e.g., measures of students' family, school, and community environments). The specific 
focus is on low-achieving students, defined as those who score in the bottom quartile 
and decile of the achievement distribution. This distinguishes this study from the 
majority of research on social context and achievement, in that most research focuses on 
mean achievement levels. The study does not focus at all on the causes of low 
achievement because NAEP data are cross-sectional and thus poorly suited to 
supporting causal inferences. 

While of value in their own right, the extensive description of the social context 
characteristics that place students at risk of scoring poorly on standardized achievement 
tests is motivated by a methodological concern: the frequent recommendations of 
researchers that NAEP and other large-scale surveys collect better data pertaining to 
students' background. Understanding who is at risk of educational failure is critical for 
designing effective educational programs and policies, and tracking the progress of 
successive cohorts of at-risk students is considered by many to be one of NAEP's most 
important functions. Changing NAEP in this regard would not be a simple task, 
however, and the descriptive information provided in this study is intended to evaluate 
both the NAEP's adequacy as currently operated and possible changes to it. 

RESEARCH QUESTIONS 

To assess the adequacy of the NAEP for reporting the test scores of the 
educationally disadvantaged, we addressed the following questions: 

• What are the distinguishing characteristics of low-achieving students? For 
instance, what population-group, family, school, and community 
characteristics differentiate low-achieving students as a group from the 
student population as a whole? 

• Which of these have independent relationships to low achievement apart 
from their relationships with other social context measures? For example, 
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are African-American students more likely to score poorly than whites from 
similar social contexts? 

• Which measures presently in the NAEP are useful for identifying the 
educationally disadvantaged and for reporting trends in their performance? 
What measures should be added to better monitor students who have low 
scores? 

• To what extent can measures currently in the NAEP serve as proxies for 
those that are omitted? 

The analyses relied on a variety of statistical techniques to provide a thorough 
description of low-achieving eighth-grade students in NAEP and NELS. The individual 
student is the unit of analysis throughout most of these descriptions. Aggregate 
variables such as the characteristics of schools and communities are included in the 
analyses, but as attributes of individual students. School-level analyses that focus on 
the characteristics of schools serving a disproportionate number of low-achieving 
students would be a valuable extension of this work but is beyond the scope of the 
current project. 

Because the NAEP has focused more on measurement of student test scores than 
on social context measures, some observers have suggested that it may lack variables 
important for describing students at risk of educational failure. In fact, our previous 
work for the TRP confirmed that the NAEP's social context measures are weaker than 
those in NELS. Since NELS contains richer measures of social context, we reanalyzed 
NELS data in this study to provide a benchmark to which the NAEP can be compared. 

CONTRIBUTIONS OF THIS STUDY 

This study is unusual in several important ways. First, and most important for 
the purposes of NCES, no studies of this sort have assessed the strengths and 
weaknesses of NAEP for describing at-risk students. Second, this study is one of the 
few that begins by defining groups of low-achieving or at-risk students — in this case, 
those scoring in the bottom decile or quartile of the test score distribution — and then 
analyzes the characteristics of those groups. Almost all studies of the relationships 
between social background and test scores describe the performance of students in a 
priori categories (e.g., African Americans and non-Hispanic whites) or model the effects 
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of background variable means or other similar statistics. Third, this study is one of a 
relatively small number that considers both individual and aggregate characteristics 
jointly (see also Kaufman and Bradby, 1992). 

FINDINGS FROM THE EXTANT RESEARCH LITERATURE 

Although much of the existing research analyzes characteristics related to 
students' average achievement scores rather than the characteristics of students in 
defined, low-achieving groups, the findings of that research provide a context for the 
results of this study. Our review of previous research focuses on the relationship 
between social context and student achievement after accounting for some other family, 
school, or community characteristics. The social context characteristics strongly associated 
with lower test scores include poorer academic backgrounds, membership in a minority 
group (e.g., blacks or Hispanics), and lower socioeconomic status (usually some 
combination of parent education, income, occupational status, and possessions in the 
home). Several characteristics are moderately associated with low scores: larger family 
size, the mother having been a teenager when her first child was bom, the child being 
primarily speaker of a language other than English, being a recent immigrant, students' 
lack of involvement in schooling activities, poor parenting styles in the home (e.g., 
authoritarian, overly permissive), school mobility, and a lack of connections with other 
adults that provide supportive social and material resources. Characteristics only 
weakly or inconsistently related to low scores include gender, living in a single parent 
household, and mother's labor force participation. 

In addition to these individual and family characteristics, researchers have 
examined whether characteristics of schools and communities are related to learning. 
The estimated net relationships between institutional characteristics and achievement 
are typically not as great as the associations between individual characteristics and test 
scores because of the difficulty of establishing aggregate, contextual effects (Blalock, 
1984). However, several school characteristics have been associated with low 
achievement scores, such as fewer opportunities to leam (e.g., poor instruction, 
placement in low tracks or ability groups), attendance at schools with lower average 
socioeconomic status, poor school climates, larger schools, and public school sector (vs. 
private). High minority enrollment and living in poor neighborhoods (e.g., high 
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unemployment levels) or different regions of the country have shown only weak or 
inconsistent net relationships to test scores. 

METHODOLOGY 

The eighth-grade samples in the 1990 NAEP and the 1988 NELS were both 
analyzed. In the case of NAEP, we analyzed the samples that were administered 
mathematics or reading in the main assessment. Information about student 
demographic, family, school, and community characteristics were obtained through the 
student and principal questionnaires. In the case of NELS, we analyzed the 
mathematics and reading scores in the base-year sample and obtained information on 
family, school, and community measures from the school, teacher, parent, and student 
questionnaires. Whenever possible, family measures in NELS were based on parent 
reports rather than student reports to improve their reliability and validity. 

The analyses involved a variety of statistical techniques. First, analyses 
compared the bottom decile and quartile on the test-score distribution to the eighth- 
grade population as a whole in terms of simple univariate statistics — that is, one 
variable at a time. Second, logistic regression analysis was used to estimate the 
independent relationships between individual's low achievement levels and social 
context characteristics in NELS, holding constant a variety of other social context 
variables. These models estimate the probability that a student with given 
characteristics will fall into the bottom quartile or decile, which can then be compared to 
the overall population probability of scoring poorly (e.g., .25 for the bottom quartile). 
Finally, we estimated several ordinary regressions and bivariate correlations among the 
social context measures themselves to further assess the adequacy of proxies in NAEP. 

RESULTS AND IMPLICATIONS FOR NAEP 

Determining the adequacy of NAEP's family, school and community measures 
depends on the purposes for which they are used. In the present analyses, we 
examined two purposes: (1) describing low-achieving students according to this 
group's family, school, and community characteristics and (2) predicting low 
achievement scores based on students' family, school, and community characteristics. It 
is noteworthy that comparisons within databases -- whether between mathematics and 
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reading tests or between the bottom quartile and decile -- were strikingly similar in both 
the univariate and multivariate analyses. 

SIMPLE DESCRIPTIONS OF LOW-ACHIEVING STUDENTS 

Simple univariate descriptions show that several of the characteristics presently 
in the NAEP are useful for the purpose of differentiating low achievers as a group from 
the eighth grade student population as a whole. Further analyses of NELS show, 
however, that the NAEP currently lacks several measures important for this purpose. 

A number of social context measures available in NAEP substantially 
differentiate low-achieving students as a group from the population. ("Substantially" 
here means there is more than a five point difference between the percentage of 
students in the bottom quartile and the percentage of the entire population consisting of 
students from a given group.) For instance, over half of the group scoring in the bottom 
mathematics quartile in NELS is white, but about three-quarters of the NELS population 
is white. Therefore, although a majority of low achievers are white, whites are 
underrepresented in the low-achieving groups when compared to the population. In 
addition, family characteristics such as low levels of parent education distinguish low- 
achieving groups from the eighth grade population. School characteristics in NAEP 
also provide informative contrasts between low achievers and the population. For 
instance, students in schools with greater percentages of minorities, with a significant 
proportion of students on free or reduced lunch, with low average levels of parent 
education, and with less amounts of homework completed by the student body are 
over-represented in the low-achieving group. 

A variety of other characteristics that differentiate low achievers from the eighth- 
grade population are absent from NAEP. These include low family income, lower 
levels of family closure (e.g., parents know very few of their child's friends), larger 
families, single parent households, and mothers who were young when they gave birth 
to the eighth grader, greater school mobility, grade retention, lower grade point 
averages, low school mean income, and low levels of closure in the school as a whole. 
NAEP could be significantly improved by adding these variables if the purpose of 
monitoring students at risk is simple description. 
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Both NAEP and NELS contain measures of region, and these show that low- 
achieving students tend to be over-represented in southern states compared to the 
population. Other than regional characteristics, both databases lack adequate measures 
of community or neighborhood environments, so it is difficult to assess whether these 
social contexts can provide instructive comparisons between low achievers and the 
population. 

Several social context characteristics that were expected to provide contrasts 
between achievement groups but do not, include mother's labor force participation, 
students' individual homework levels, English as a second language, immigrant status, 
and school sector. 

PREDICTING LOW ACHIEVEMENT SCORES 

Because many of these social context measures are strongly related to each 
other, their univariate relationships with low achievement can be quite different from 
their independent effects on achievement when other aspects of social context are held 
constant. Therefore, not all of the measures useful for the purpose of describing low- 
achieving students are helpful for predicting the likelihood that individual students will 
score poorly on achievement tests. There is both overlap and inconsistency between the 
lists of measures useful for both purposes. 

When one controls for a large number of social context variables, numerous 
variables that show strong univariate relationships with low achievement do not show a 
large independent association with low achievement. For instance, measures for single 
parent households, school minority composition, and percentage in the school on free 
lunch differentiated low-achieving groups from the population in the univariate 
descriptions, but they were not substantially independently related to low achievement 
scores. For example, students in primarily black schools are barely more likely to score 
poorly than students in primarily white schools once a variety of other variables 
(including the students' own population group) are taken into account. This finding 
contrasts the univariate descriptions which revealed sharp differences between the 
percentage of low achievers and the population who attend more racially and ethnically 
diverse schools. In some cases, the lack of independent relationships simply reflects the 
correlations among the many social context measures used in the analyses; once enough 
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are included, the independent associations between many of them and low achievement 
will be small, even if there is in fact a causal link between them and achievement. 

However, several demographic, family, and school characteristics presently in 
the NAEP continue to be related to low achievement scores, even after holding constant 
a large number of other social context characteristics. Population group membership 
and parents' educational attainment are substantially related to low scores both before 
and after other family, school, and community differences have been taken into account. 
Black students are much more likely to score poorly than Hispanics and especially 
whites from similar social contexts. In addition, while students whose mothers or 
fathers only graduated from high school have probabilities of low achievement almost 
equal to the population, eighth graders who have college-educated parents are much 
less likely to score in the bottom mathematics quartile even after other factors have been 
taken into account. Accounting for other differences in social context, there are striking 
differences between students in schools that have highly educated parents and those 
that do not. In addition, students who attend schools that assign greater amounts of 
homework are less likely to be low achievers. 

Some of the social context variables that are included in NELS, but not NAEP, 
have substantial independent relationships with the probability of being a low achiever. 
The most important characteristics absent from the NAEP include measures for grade 
retention and immigration status. A student who is held back has about a forty percent 
chance of being a low achiever when other characteristics have been controlled. 
Immigration status, both of the student and the eighth grader's mother, are strongly 
related to test scores, but in the opposite direction of what is often assumed. For 
instance, controlling for other differences in social context, students who are immigrants 
(or whose mothers were bom outside the U.S.) have a fifteen percent chance of scoring 
in the bottom mathematics quartile. 

A number of other characteristics absent from the NAEP either are not 
statistically related to low scores or have only weak relationships to it. These include 
family income; family size; mother's age at birth of the eighth grader; family closure; 
school mobility; and school income, closure, and size. For instance, there is a only very 
weak relationship between scoring poorly in mathematics and family income after other 
differences in social context have been taken into account. 

viii 
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NAEP PROXIES FOR SOCIAL CONTEXT MEASURES 



These findings raise the question: how adequate are the social context variables 
in NAEP as proxies for measures absent from it (e.g., income)? The answer differs 
according to the purpose of the NAEP. If the goal is only to predict the probability that 
a student will be a low achiever, our analyses suggest that a large enough number of 
the NAEP social context variables, taken together, are a fairly good proxy for omitted 
measures such as income. That is, adding an income variable to them will not 
dramatically change the predicted probability of scoring in the bottom quartile. 

However, further analyses show that for other purposes — for example, to 
answer the question, what are the achievement scores of poor students? — the variables 
included in the NAEP do not provide an adequate proxy for income. For example, 
regressing the family income measure on the other social context characteristics 
available in NAEP results in an adjusted R2 of .31. This means that the other social 
context variables predict only about a third of the variance in income. Because family 
income has a strong unadjusted relationship with low test scores and a weak 
relationship once other factors are controlled, we conclude that the portion of family 
income that matters in predicting the probability of low achievement is the third of the 
variance in income that is related to the other social context variables in our full NELS 
model. 

Among the school measures, the relationships are much higher, but even here 
existing NAEP measures do not provide good proxies for school poverty or school 
closure (i.e., social connections among families in the school). At most, two-thirds of the 
variance of these measures is explained by other demographic, family, school, and 
community characteristics that are available in NAEP. Because a large portion of the 
variance is left unexplained, the current NAEP does not provide sufficient proxies for 
these omitted school variables if the purpose is to identify the same groups of students. 

In short, if the purpose of the NAEP is to predict low achievement scores, the 
extant NAEP is not affected by the omission of some particular social context measure. 
However, if the aim is to allow researchers and policymakers to identify the same low 
achieving students, then the NAEP currently contains inadequate proxy measures. 
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INTRODUCTION 



Students are placed at risk for educational failure for many reasons, many 
involving students' experiences in their families, schools, and communities. Recent 
studies have focused criticisms on families and schools and their inability to effectively 
provide a quality education, especially for disadvantaged students. Some argue that we 
are in such deep trouble that the U.S. needs to systematically restructure the 

educational system, not just schools (see Murphy, 1991; Smith and O'Day, 1991). In 

\ 

fact, recent federal legislation was aimed at providing an overall vision to provide 
federal support for local state, community, and school systemic educational reform 
(Smith, 1995). 

Understanding who is at risk of educational failure and why is critical for 
designing effective programs and public policy. The costs of not doing so are great. 
Natriello et al. (1990: 40) warn that "Failure to educate the educationally disadvantaged 
adequately may have catastrophic consequences for the social and economic well-being 
of this country." For example, there are high costs imposed on society — forgone tax 
revenues, earnings, and economic output; greater welfare and medical expenditures; 
and greater costs of crime and incarceration (Levin, 1972, 1985, 1986, 1989; Catterall, 
1985; McDill et al., 1986; Bishop, 1989). Since the educationally disadvantaged 
population may be expanding, these costs to society may soon become even more 
severe (Natriello et al., 1990; Pallas et al., 1989). 

Because the meanings of the terms "at risk" or "educationally disadvantaged" 
are constantly shifting -- as is typical for rhetorical devices that are emotionally charged 
(Gans, 1990; Jencks, 1991; Wilson, 1991) — it is critical to provide a definition at the 
outset. Generally, at-risk students are defined as those who are unlikely to succeed in 
the present school system because they do not have the experiences in the home, 
family, school, and community on which educational success is built (Levin, 1994). For 
the specific purposes of this study, we focus on low-achieving students, defined as 
those who score in the bottom quartile or decile of the achievement distribution. 
Therefore, this study differs from the majority of the research on student achievement 
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that examines average achievement levels among different groups of students— for 
example, the many studies that explore the relationships between SES and mean 
achievement. 

To better understand the indicators of risk related to student achievement, 
researchers have recommended that the National Assessment of Educational Progress 
(NAEP) and other large-scale surveys collect better data pertaining to students' social 
contexts (e.g., Grissmer et al., 1994). In fact, tracking the progress of successive cohorts 
of at-risk students is considered by many to be one of NAEP's most important 
functions. Changing NAEP in this regard would not be a simple task, however, and the 
descriptive information created in this study is intended to evaluate both the adequacy 
of the NAEP as currently operated and possible changes to it. 

Specifically, the study is intended to assess the adequacy of the NAEP for 
describing students at risk of educational failure in terms of the various social contexts 
in which they live (e.g., measures of students' family, school, and community 
environments). It is a descriptive examination of both the institutional factors (e.g., 
school and community) and the individual characteristics (demographic and family) 
that place students at risk of scoring poorly on standardized achievement tests. The 
study does not focus at all on the causes of low achievement because NAEP data are 
cross-sectional and thus poorly suited to supporting causal inferences. 

While separating the unique contribution of individual and institutional factors 
is a complex exercise (Sewell and Hauser, 1993; Gamoran, 1992), better data will 
contribute to a more thorough understanding of what factors are related to risk of low 
achievement. Such data will help avoid the traditional pitfalls of some educational 
research that adopts a "deficit model" of educational failure, placing the blame mainly 
within the student and promote work that examines how institutional characteristics 
are related to educational failure (Rumberger, 1994; Delgado-Gaitan, 1991; Wehlage et 
al., 1989; Bronfenbrenner, 1979). 



DRAFT 

Identifying Students at Risk 



2 



Berends & Koretz 



RESEARCH QUESTIONS AND APPROACH 

To assess the adequacy of the NAEP for reporting the test scores of the 
educationally disadvantaged, we address the following questions: 

• What are the distinguishing characteristics of low-achieving students? For 
instance, what population-group, family, school, and community 
characteristics differentiate low-achieving students as a group from the 
student population as a whole ? 1 

• Which of these factors have independent relationships to low achievement 
apart from their relationships with other social context measures? For 
example, are African-American students more likely to score poorly than 
whites from similar social contexts? 

• Which measures presently in the NAEP are most useful for identifying the 
educationally disadvantaged and for reporting trends in their performance? 
What measures should be added to better monitor students who have low 
scores? 

• To what extent can measures currently in the NAEP serve as proxies for 
those that are omitted? 

Because the NAEP has focused more on measurement of student test scores than 
on social context measures, some observers have suggested that it may lack variables 
important for describing students at risk of educational failure. Indeed, our previous 
work for the TRP confirmed that the NAEP's background variable set is substantially 

1 Consistent with our other work (Berends and Koretz, in press; Barron and Koretz, in press), 
we often use the term "population group" to refer to groups commonly called "racial" or 
"ethnic." We do this because the classifications in question are not unambiguously racial or 
ethnic. Rather, they are socially conventional categories that rest in part on racial and/or ethnic 
differences. For example, Americans who are of mixed white and black ancestry are generally 
classified as black as a matter of social convention. Similarly, some Caucasian South Asians are 
classified as "Asian," and many Hispanics with substantial Native American ancestry are 
conventionally classified as "white." We simply accept the socially conventional categories as 
given and use the neutral and less misleading term "population group" to refer to them. 
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weaker than that of NELS (Berends and Koretz, in press). Since NELS contains richer 
measures of social context, we reanalyzed NELS data in this study to provide a 
benchmark to which the NAEP can be compared. 

We examine these questions using a variety of statistical techniques to provide a 
thorough description of low-achieving eighth-grade students in NAEP and NELS. The 
individual student is the unit of analysis throughout most of these descriptions. 
Aggregate variables such as the characteristics of schools and communities are included 
in the analyses, but as attributes of individual students. School-level analyses that focus 
on the characteristics of schools serving a disproportionate number of low-achieving 
students would be a valuable extension of this work but is beyond the scope of the 
current project. 

The analysis consists of three steps: First, we extensively review the literature 
on the individual and institutional characteristics that place students at risk of low 
achievement scores. While any such review is selective, most of the studies included 
are those that analyze national data to examine the independent effects of demographic, 
family, school, and community measures on academic achievement (in most cases, 
average achievement). Second, we conduct our own analysis of NAEP and NELS to 
highlight those characteristics that distinguish low achievers from the eighth-grade 
student population as a whole. Finally, we jointly examine social context characteristics 
to ascertain which are important independent predictors of low achievement scores 
using logistic regression techniques. 

CONTRIBUTIONS OF THIS STUDY 

Although there have been many studies of the student characteristics that are 
associated with one or another measure of school failure, this study is unusual in 
several important ways. Most important for the purposes of NCES, no studies of this 
sort have assessed the strengths and weaknesses of NAEP for describing at-risk 
students. Almost all studies of the relationships between social background and test 
scores start with the background characteristics, either describing the performance of 
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students in a priori categories (e.g., African Americans and non-Hispanic whites) or 
modeling the effects of background variables on mean outcomes. This study, in 
contrast, is one of the few that begins by defining groups of low-achieving or at-risk 
students — in this case, those scoring in the bottom decile or quartile of the test score 
distribution -- and then analyzes the characteristics of those groups. Moreover, when 
examining the effects of social context measures on student achievement, this study is 
unique in investigating the independent effects on low test scores rather than average 
achievement levels. This study is also one of a relatively small number that considers 
both individual and aggregate characteristics jointly (see also Kaufman and Bradby, 
1992). Finally, it provides an extensive review of the research on social contexts that 
that place students at risk of low test scores for the specific purpose of improving 
measures in the NAEP. 



1 
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FINDINGS FROM THE EXTANT RESEARCH LITERATURE 



Although much of the existing research analyzes the average achievement of 
students rather than the characteristics of students in defined, low-achieving groups, 
the findings of that research provide a context for the results of this study. The 
following section briefly highlights several demographic, family, school, and 
community measures shown in previous research to be related to student achievement 
after accounting for some other family, school, or community characteristics. 

INDIVIDUAL AND FAMILY CHARACTERISTICS 

Much of the work on low-achieving students stresses the importance of 
individual factors, especially demographic and family characteristics. The following 
sections review the research on the relationships of test scores to population group, 
gender, academic background, family income, parents' educational attainment, single 
parent households, family size, age of mother at birth of the child, mother's 
employment status, immigrant status, and English as a second language. 

Minority Population Group 

Research has consistently shown that minority students are more likely to score 
poorly on standardized tests than nonminorities. While their are several explanations 
for why minorities continue to score poorly (Delgado-Gaiton, 1991, 1992; Wilson, 1987, 
1991, 1993; Ogbu, 1979, 1989, 1992; Omni & Winant, 1994; McCarthy, 1990), minorities 
have made significant progress over the past twenty-five years in closing the minority- 
nonminority test score gap (Koretz, 1986, 1992; Linn and Dunbar, 1990; -Smith and 
O'Day, 1991a; Grissmer et al., 1994; Mullis, Dossey, Campbell, Gentile, O'Sullivan, and 
Latham, 1994). While the gap has closed, the average achievement gap between blacks 
and whites remains large; it varies across tests, grades, and subject areas, but it is often 
about .75 of a standard deviation in NAEP and NELS, (Berends and Koretz, in press). If 
black and white students are held equal on a wide range of family, school, and 
community measures, the gap is reduced substantially. Yet, the adjusted gap remains 
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about .40 of a standard deviation. Similarly, the unadjusted gap between Hispanics and 
whites is about .60 of a standard deviation, while the gap that adjusts for several social 
context measures is about .25 of a standard deviation (see Berends, Koretz, and Lewis, 
1994) 

Gender 

Generally, girls do better in school than boys until puberty (Klein, 1985). Several 
argue that the learning environment and organization of junior high and high schools is 
better suited to boys than girls (Eccles and Hoffman, 1987; Steinkamp and Maehr, 

1984), but the empirical evidence for such claims is lacking (see Mickelson, 1989). After 
the onset of puberty through the high school years, girls tend to outperform boys in 
verbal tasks, but boys score higher on tests measuring visual-spatial and quantitative 
skills. Studies that control for other family, school, and community factors find that 
high school girls score lower than boys on mathematics tests by about .10 of a standard 
deviation, score higher on writing by about .30 of a standard deviation, and do not 
differ from boys on reading or vocabulary tests (Gamoran, 1987). While the gender gap 
in writing achievement continues to favor girls over boys, the boys advantage in 
mathematics may be dissipating (see U.S. Department of Education, 1994). 

Past Academic Success 

One of the strongest predictors of future success in school is past success. 
Longitudinal studies that control for prior achievement levels (e.g., test scores, grades, 
retention in grade, mobility among schools) find students' past academic record to be a 
strong predictor of subsequent test scores (e.g., Alexander and Pallas, 1985; Coleman 
and Hoffer, 1987; Gamoran, 1987; Lee and Bryk, 1989; Hoffer, 1992; Gamoran et al., in 
press). Moreover, being held back in school and changing schools often is detrimental 
to student learning (Finn, 1989; Kaufman and Bradby, 1992). In fact, much of the debate 
about the true independent effects of family, school, and community measures on test 
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scores centers, in part, on researchers' capabilities of fully accounting for past academic 
success (see Jencks, 1985; Slavin, 1990). 



Family Income 

In studies of academic achievement, family income is often incorporated into a 
composite defined as socioeconomic status (SES). 2 Some studies suggest that SES is the 
strongest predictor of student achievement than SES (see Coleman et al., 1966; Jencks et 
al, 1972, 1979; Gamoran, 1987; Lee, Bryk, and Smith, 1993). Socioeconomic status may 
be a proxy for a variety of family processes (ability, culture, tastes, stimulating 
environments, parenting styles and involvement in child's education, educational 
involvement). 3 Whatever the underlying factors, SES remains as strong predictor of 
student achievement. 

The findings are consistent in the few studies that examine the relationship 
between achievement and family income by itself: student in higher income families 

2 Many studies that analyze data from the U.S. Department of education rely on composites in 
these databases for SES, usually an unweighted linear combination of parents' educational 
attainment, parents' occupational status, family income, and measures of home possessions. 

Such scaling, however, may introduce measurement error and thus attenuate associations of SES 
with student outcomes. In addition, data since the National Longitudinal Survey of the High 
School Class of 1972 (NLS-72), through High School and Beyond (HSB) and the base year 1988 
National Education Longitudinal Survey (NELS) rely on Duncan's Socioeconomic Index (SEI) 
(Duncan, 1961), but the SEI is based on the 1960 Census. Therefore, the occupational status 
components of the SES composites in HSB and NELS are dated and of little use. (The 1992 wave 
of NELS, however, is the exception since it updates the SEI with the 1990 Census.) 

3 Examples of research on these various processes underlying parent educational attainment 
include: Plomin (1986) on ability; Bourdieu and Passeron (1977) and Bourdieu (1984) on culture 
and tastes; Kohn and Schooler (1983), Bradley (1985), Parcel and Menaghan (1990), and 
Menaghan and Parcel (1991) on stimulating home environments; Baumrind (1968, 1978) and 
Dombusch et al. (1987), and Steinberg et al. (1989) on authoritarian, authoritative, and 
permissive parenting styles; and Stevenson & Baker (1987), Lareau (1989), Epstein (1990), 

Astone & McClanahan (1991), and Schneider and Coleman (1993) on parent involvement in 
child's education. 
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tend to have higher achievement scores. For example. Hill and O'Neill (1994) find an 
increase of $10,000 per year is associated with an increase in scores of 2.4 percentile 
points. However, it may be important to measure income over a period of years since 
Hanushek (1992) finds that such a measure (i.e., "permanent income") is positively 
related to achievement, while current income measured at one point in time is not. 

Educational Attainment of Parents 

Like income, parents' educational attainment is an indicator of socioeconomic 
status and may be a proxy for several underlying family processes. Moreover, parents' 
education levels are also strongly related to student achievement in those studies that 
include the attainment measures separately, rather than subsuming it under the more 
global measure of family socioeconomic status (SES). 

Students with mothers who have more years of schooling tend to score better on 
achievement tests than students who have less educated mothers. For example, 
previous analyses of NELS data show that students whose mothers graduated college 
score about .33 of a standard deviation higher in reading (.38 of a standard deviation in 
mathematics) than students whose mothers graduated high school (Grissmer et al., 

1994; see also Hill and O'Neill, 1994). 

There is also empirical evidence that father's educational attainment is 
associated with achievement even after other family characteristics are taken into 
account (e.g., family income and mother's education). For instance, Grissmer et al. 
(1994) find that students with college-educated fathers score about one-third of a 
standard deviation above those whose fathers did not go beyond high school. This 
finding was consistent across mathematics and reading tests in NELS and the National 
Longitudinal Survey of Youth (NLSY). 

Single Parent Households 

It has frequently been argued that children in single parent households may be 
shortchanged in terms of both money and time and thus may tend to perform more 
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poorly in school (McLanahan and Sandefur, 1994). Research, however, is inconsistent 
in terms of the relationships between student test scores and the number of parents in 
the household. 4 

Several researchers find that living in a single parent household is associated 
with lower student achievement. Hetherington, Camara, and Featherman (1981) in 
their comprehensive review, find consistent, yet small, differences in achievement 
favoring children from two-parent families. Analyzing the mother-child data from the 
National Longitudinal Survey of Youth, Krein and Beller (1988) find that the negative 
effect of living in a single-parent family on achievement scores increases with the 
number of years spent in this type of family and is particularly influential during the 
preschool years. 

These relationships may be due to other factors, however. For example, Milne, 
Myers, Rosenthal, and Ginsburg (1986) find that although the total differences are fairly 
substantial between students in single- and two-parent families (about nine percentile 
points), these differences can be explained by other family factors such as income, 
mother's employment, parental expectations, and parental help with homework. 

Others analyzing different data have found similar results (Desai et al., 1989; Hanushek, 
1992; Mulkey, Crain, and Harrington, 1992; Hill and O'Neill, 1994; Grissmer et al., 1994) 

Family Size 

There is a substantial amount of research that shows that larger family size is 
associated with lower test scores (see Blake, 1989; Alwin, 1991; Hanushek, 1992; Hill 
and O'Neill, 1994). Blake's (1989) main hypothesis explaining these differences is that a 
greater number of siblings dilutes familial resources, thus lowering the achievement 
levels of students in such large families. The dilution occurs across a variety of familial 
resources, including parental time, emotional and physical energy, attention, and ability 
to interact with children as individuals, and in the amount of financial resources 

4 This contrasts with the more consistent findings about the effects of household structure on 
other schooling outcomes (see McLanahan and Sandefur, 1994). 
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allocated to each child. Some have argued that family size effects are not significant 
because other parental characteristics have been ignored in research on family size (e.g., 
socioeconomic status, parental IQ, personality characteristics, perceptions of desirable 
qualities in children) (Lindert, 1977). 

However, Blake (1989) argues that genuine family size effects exist, and finds 
that they are significant in many cases. Other research has supported her findings. For 
example, Hanushek (1992) shows that, as family size increases, the achievement growth 
of each child in the family falls (but at a declining rate). He also estimates that changes 
in family size that occurred over the past two decades help explain half or more of the 
aggregate changes in some student test score trends between 1965 and 1985, although 
this may be an overstatement (see Koretz, 1986). 

Age of Mother at Birth of First Child 

Young mothers may be in their teenage years, children themselves, unmarried, 
less educated, poor, and have little knowledge or skills to parent a young child — all of 
which are related to lower test scores. Research on the effects of early childbearing on 
student achievement is mixed. Moore and Snyder (1991) find that mother's age at birth 
was less important as a predictor of test scores than the mother's score on a test of 
cognitive achievement (also see Hill and O'Neill, 1994; Desai et al., 1989). The 
association between early childbearing and low achievement scores has been fairly 
small in multivariate models, particularly when compared to other characteristics of the 
mother (e.g., income, educational attainment, and cognitive ability) (Grissmer et al., 
1994; Moore and Snyder, 1991; Belmont, Cohen, Dryfoos, Stein, and Zayac, 1981). 

Mother's Labor Force Participation 

The relationship between maternal employment and student achievement has 
been the subject of debate. While some argue that maternal labor force participation 
decreases the amount of time available for each child and increases the stress in the 
family, others argue that employment enhances a woman's self-esteem, which transfers 
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into fostering a better home environment (Hoffman, 1989). Although increased 
maternal self-esteem may explain part of the association between maternal employment 
and test scores, the increase in family income due to mothers working for a wage may 
be the key factor because of the increased financial resources that may benefit student 
learning opportunities. 

The research on the correlates of mother's employment is inconsistent and also 
shows varying relationships depending on other variables. For example, Milne, Myers, 
Rosenthal, and Ginsburg (1986) find that mother's labor force participation is negatively 
associated with achievement; the more the mother works, the more negative the 
relationship. However, analyzing longitudinal data, Gottfried, Gottfried, and Bathurst 
(1988) did not find a negative relationships between maternal employment status and 
several different achievement tests. Others have found that the effects of mother's 
employment status differ according to the age of the child, with the effect being 
negative for very young children and negligible or slightly positive for older children 
(Desai et al., 1989; Blau and Grossberg, 1992). 

Immigrant Status and English as a Second Language 

Immigration and limited English proficiency are issues that have become 
increasingly important policy issues, especially because of their implications for public 
education (McDonnell and Hill, 1993; Vemez and Ronfeldt, 1991; Natriello et al., 1990). 

The percentage of the U.S. Population who primarily speak a language other 
than English has increased significantly in the last ten years. For example, in 1979 
about nine percent of the U.S. population primarily spoke a language other than 
English; in 1989, the percentage rose to twelve percent. The high levels of immigration 
in the past few decades contributed significantly to this increase (McArthur, 1993). 

Most of recent immigrants are concentrated in a few areas of the country, such as 
California, New York, Florida, Texas, and Illinois (McDonnell and Hill, 1993), so some 
schools serve populations in which the percentage of children whose primary language 
is not English is far higher than the national average. 
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When considering students at risk of educational failure, policymakers, 
researchers, and the public have focused their attention more on the Hispanic 
population group than on other immigrant population groups such as Asians. The 
reason for this may be that Asians typically have better educational outcomes, 
especially test scores, than Hispanics (Rumberger, 1987, 1994; U.S. Department of 
Education, 1994; Caplan, Choy, and Whitmore, 1991, 1992; Stevenson and Stigler, 1992). 

The independent effects of immigrant status or language use on lower mean test 
scores is not well-established. For instance. Hill and O'Neill (1994) find that when 
mother's primarily speak a language other than English, the child's test scores tend to 
decrease, the relationship is explained by other family and community characteristics. 
This finding is consistent with others who have argued that other family and schooling 
factors are more critical to student learning than immigrant status and non-English use 
(Caplan et al., 1991; Delgado-Gaitan, 1991). 

INSTITUTIONAL CHARACTERISTICS 

Researchers have found it difficult to establish unambiguously the associations 
between students' school or community characteristics and test scores (Gamoran, 1992; 
Lee et al., 1993). The main reason for this is that most of the variation in student 
achievement scores lies within schools rather than between them, and only recently 
have methodological techniques advanced to separately estimate effects at the 
individual and school levels (see Bryk and Raudenbush, 1992). 5 

Often, researchers confuse the "school, an organization, with schooling, a 
process that individual students experience" (Bidwell and Kasarda, 1980, p. 402). While 
important, schooling processes, such as classroom instruction, are difficult to quantify 

5 While muti-level modeling techniques (see Bryk and Raudenbush, 1992; Diprete and Forristal, 
1994) allow researchers to model the separate of effects of individual-level and school-level 
variables, these techniques have not yet been sufficiently developed for estimating models with 
dichotomous dependent variables such as those analyzed in this study (e.g., bottom decile and 
quartile of the test score distribution). 



DRAFT 

Identifying Students at Risk 



13 



Berends & Koretz 



with reliable and valid measures (Gamoran et al., in press; Hoffer and Moore, in press; 
Porter, 1991; Gamoran, 1989), so we limit our review to those studies that examine the 
relationships between student test scores and more global school organization 
characteristics — composition, size, sector, and climate. We also discuss research on the 
effects of students' neighborhoods and communities on school achievement. 

Socioeconomic Composition 

Schools with higher proportions of student from high socioeconomic 
backgrounds have higher achievement, higher graduations rates, and more college- 
bound graduates. This has been known for some time (Wilson, 1959; Conant, 1961). 

The question of interest, however, is whether these relationships reflect contextual 
effects above and beyond the individual-level relationships between SES and 
achievement. For example, one possible contextual effect would arise if schools with 
higher numbers of socially advantaged students provide environments that foster 
success over and above the effects of individual background characteristics (Gamoran, 
1992). 

This topic has received considerable attention over the past thirty years, and 
generally the research reveals that while students scores may be higher in schools that 
have greater percentages of higher status students, the net effect of school SES on 
achievement is not as strong as individual effects of SES on achievement (Gamoran, 
1992). For example, the landmark Coleman report (Coleman et al., 1966) reports that 
schools with higher average socioeconomic levels (SES) also have higher student test 
scores, even after controlling for individual characteristics. Analyses of High School and 
Beyond show that school SES is strongly related to mathematics scores (Bryk and 
Driscoll, 1988; Lee and Bryk, 1989), but school SES does not have significant 
independent effects on other subject matter scores (Gamoran, 1987). Other studies also 
show a weak relationship between student achievement and school SES after 
accounting for individual characteristics (Alexander et al., 1979; Alwin and Otto, 1977). 
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Racial Composition 

Another finding of Coleman et al. (1966) is that the achievement of minority 
students is higher in racially integrated schools even after controlling for individual and 
other school and community characteristics. However, similar to other school effects, 
minority composition is not as strongly related to student achievement scores when 
compared to the strong net effects of individual measures (e.g., population group and 
socioeconomic status). Gamoran (1987) finds that students in schools with a greater 
proportion of black students score worse on science and vocabulary tests; however, he 
finds the Hispanic composition of schools has no significant relation to the six tests 
administered in High School and Beyond data. More recently, Entwisle and Alexander 
(1992) show that, for younger children, family economic factors far outweigh the 
influence of the racial mix of the school. Generally, these findings are consistent with 
reviews of the short-term effects of school desegregation that reveals mixed, yet mostly 
positive, effects of school desegregation on minority students' achievement scores 
(Mahard and Crain, 1983; Crain and Weisman, 1972). 6 

School Size 

There are two arguments about the relationship between school size and student 
achievement (see Lee et al., 1993). First, from the viewpoint of efficiency, some argue 
that larger schools are more cost-efficient in that they can spread costs over a larger 
pupil base and offer more resources (Chambers, 1981). Others, argue that activities in 
large schools are more formalized and specialized, which contributes to a climate 
characterized as bureaucratic, alienating, and isolating (Newmann, 1981; Bryk et al., 
1990; Lee et al., 1993). 

Some research has shown a negative relationship between school size and 
student test scores, apparently because school size influences achievement through 

6 Compared to the short-term effects of desegregation on achievement, studies of the long-term 
effects consistently show that desegregation is related to positive outcomes (see Wells and 
Crain, 1994). 
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other schooling processes. For example, Bryk and Driscoll (1988) find a negative 
relationship between larger schools and mathematics achievement, controlling for 
student and school characteristics. This association is explained by larger schools being 
more communal. That is, the smaller schools have shared values about the educational 
mission, a less diversified curriculum for students, a teaching staff that is committed 
and collegial, and teachers who go beyond their defined roles to help students learn and 
adjust socially (see also Bryk, , Lee, and Holland, 1993). 

School Sector 

It is generally accepted that public schools are outperformed by Catholic 
schools; however, the magnitude of these effects and their implications are often the 
center of heated debate. One of the more balanced perspectives is Jencks' (1985), who 
states that students learn slightly more in the Catholic sector than the public over the 
last two years of high school; however, the magnitude is uncertain. 7 

In addition to the average effects of school sector on the general population of 
high school students, there is some research that shows Catholic schools benefit 
educationally disadvantaged students (Coleman, Hoffer, and Kilgore, 1982; Coleman 
and Hoffer, 1987; Lee and Bryk, 1989; Bryk et al., 1993). However, some argue that "the 
evidence that Catholic school are especially helpful for initially disadvantaged students 
is quite suggestive, but not conclusive" (Jencks, 1985, p. 134). 

While the size of the effects of school sector are often debated, more recent 
research attempts to understand how these effects occur through the internal 
organization of schools (Bryk et al., 1990; Lee et al., 1993). A growing number of studies 
have compared the organization of private to public schools in terms of ability grouping 
and tracking, social relationships between teachers and students, parent involvement, 
and normative order of the school (Bryk and Driscoll, 1988; Lee and Bryk, 1988; Bryk et 

7 This uncertainty is likely to remain since it is very difficult to control for problems of selection 
when comparing Catholic to public schools, despite various innovative attempts (see 
Goldberger and Cain, 1982; Coleman and Hoffer, 1987; Sander and Krautman, 1995). 
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al., 1993). Many of the positive effects of the Catholic schools on disadvantaged 
students are explained by these organizational factors. 

Neighborhoods and Communities 

Neighborhood or community characteristics may have an impact on schooling 
outcomes such as academic achievement (Corcoran and Danziger, in press; Brooks- 
Gunn et al., 1993; Garner and Raudenbush, 1991). For example, Garner and 
Raudenbush (1991) found that lower quality neighborhoods had a negative impact on 
student achievement. Their neighborhood measure included such factors as proportion 
of single parent and large households, unemployment rates for adults and youth, and 
overcrowding. However, others have not found significant neighborhood and 
community effects once other family and school characteristics have been taken into 
account (Gamoran, 1987; Hoffer 1992). 

The mechanisms through which neighborhood effects on test scores operate are 
mainly speculative at this point because of the difficulty in specifying models that 
account for the family, school, and community contexts that influence student learning 
(Manski, 1994; Sewell and Hauser, 1993; Brooks-Gunn et al., 1993; Gamoran, 1992; 
Mayer & Jencks, 1989; Hauser, 1970). 8 One important reason for this problem is the lack 
of data rich enough to provide decent measures of family, school, and community 

8 The mechanisms most often dted are (1) contagion theories suggesting that once peer influence 
passes a certain threshold detrimental effects on schooling outcomes result (Crane, 1991a, 

1991b); (2) collective socialization theories hypothesizing that neighborhood role models and 
monitoring are important for child development (Coleman, 1988; Coleman and Hoffer, 1987; 
Wilson, 1987); (3) competition theories postulating that neighbors compete for scarce resources 
within the neighborhood; and (4) theories of relative deprivation hypothesizing that individuals 
evaluate their situation or relative standing compared to their neighbors (Meyer, 1970). The first 
two theories predict positive outcomes for children if the neighbors are affluent; competition or 
deprivation theories predict that affluent neighbors may contribute to negative outcomes (for 
further elaboration see Mayer and Jencks, 1989). The evidence for or against any one of these 
theories is far from clear. 
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factors. Often data are rich in two or three of these social contexts, but seldom all three. 
For example, most data from the Department of Education (HSB or NELS) provide 
good data on family and school characteristics, but they do not contain quality 
measures of neighborhood characteristics. By contrast, data often used to analyze 
poverty issues such as the Panel Study of Income Dynamics (PSID) or the National 
Longitudinal Survey of Youth (NLSY) provide an abundance of data on family and 
neighborhood /community characteristics. School measures, however, are severely 
lacking in these databases (see Danziger, Sandefur, and Wienberg, 1994; Maier, 1991). 

Region of the Country 

Students' schooling outcomes have differed by region of the country for some 
time (Duncan, 1968; Hauser and Featherman, 1976; Featherman and Hauser, 1978; 
Farley and Allen, 1987). 

Like other schooling outcomes, test scores have differed by region with those 
students living in the south scoring slightly lower than those living in other areas of the 
United States. For example, NAEP has consistently shown lower scores in the south 
than in other regions (U.S. Department of Education, 1991). Some mutivariate analyses 
show that much of this regional difference may be attributed to family or school 
characteristics. For example, Grissmer et al. (1994) show that students living in the 
South score less than .10 of a standard deviation on mathematics test below student 
living in the Northeast, after controlling for various family characteristics. Gamoran 
(1987) finds that students in the South score more poorly on mathematics and 
vocabulary tests than students in the Northeast, but many of these differences 
disappear when school characteristics are taken into account. 
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SUMMARY: INDIVIDUAL AND INSTITUTIONAL CHARACTERISTICS 

For our purposes, we summarize the extant research reviewed above in terms of 
what characteristics place students at risk of educational failure. The relationships 
between low achievement scores and the individual, family, school, and community 
characteristics are categorized according to whether they are strong, moderate, or weak 
or inconsistent after controlling for some other family, school, or community characteristics. 
While these categories are subjective, they are intended to illustrate the range of the 
relationships that have been established in the extant research. 

Stronger associations with low test scores: 

• membership in a minority group (e.g., blacks or Hispanics) 

• poorer academic backgrounds (prior low achievement levels, held back in 
school) 

• lower parental education levels 

• lower family income 

• lower parents' educational expectations 

• lower socioeconomic status (usually some combination of parent education, 
income, occupational status, and possessions in the home) 

More Moderate associations with low test scores: 

• larger family size 

• mother was a teenager at birth of first child 

• primarily non-English speaker 

• immigrant 

• students' lack of involvement in schooling activities 

• poor parenting styles in the home (e.g., authoritarian, overly permissive) 

• school mobility 

• a lack of connections with other adults that provide supportive social and 
material resources 

• lower socioeconomic status school composition 
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• poor school climates (e.g., inconsistent educational vision, low expectations, 
a lack of school leadership, lack of school order and discipline, low levels of 
parent involvement) 

• larger schools 

• school sector (e.g., public schools compared to Catholic or other private 
schools) 

• fewer opportunities to learn (e.g., poor instruction, placement in low tracks 
or ability groups) 

Weak or inconsistent associations with low test scores 

• gender (e.g., girls score worse on tests measuring quantitative and spatial 
reasoning skills, while boys score worse on tests measuring verbal ability) 

• single parent household 

• mother works full- or part-time outside the home 

• greater minority school composition (especially for minority students ) 9 

• poorer neighborhoods (e.g., high unemployment levels) 

• rural or certain urban communities 

• region of the country (e.g.. South) 



9 Note that this list refers to characteristics that have associations with low test scores after 
accounting for other social context variables. On average, schools with high minority 
enrollments have substantially lower mean achievement than do other schools. However, the 
question addressed here is whether the minority enrollment of schools has an association with 
low scores after taking into account factors that include the population group of the students - 
for example, whether African American students in schools with high minority enrollments 
have lower scores than do African American students with similar backgrounds but in low- 
minority schools. 
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3. 



DATA AND METHODS 



We analyzed the eighth grade samples in the 1990 NAEP and the 1988 NELS. In 
the case of NAEP, we analyzed the samples that were administered mathematics or 
reading in the main assessment. The mathematics and reading samples are 
independent in the NAEP; that is, the same students in the main assessment do not take 
both the reading and mathematics test. We used information about student 
demographic, family, school, and community characteristics obtained through the 
student and principal questionnaires (for more details see Johnson & Allen, 1992). 

In the case of NELS, we analyzed the mathematics and reading scores in the 
restricted-use base-year sample (for details see Ingels et al., 1990). In comparison to 
NAEP, NELS administered reading, mathematics, science, and history tests to the same 
sample of students. NELS obtained information on family, school, and community 
measures from school, teacher, parent, and student questionnaires. Whenever possible, 
we used family characteristics based on the parent responding to the parent survey 
rather than student reports to improve the reliability and validity of these measures (see 
Kaufman & Rasinski, 1991). 

MEASURES IN ANALYSES 

We examined a large number of measures related to low achievement in 
mathematics and reading, including demographic, family, academic record, school, and 
community characteristics. The definitions for the variables used in the analyses appear 
in Tables 3.1 through 3.4. Table 3.1 shows the measures that were common to both 
NAEP and NELS; Table 3.2 shows the language use characteristics in both databases, 
but operationalized differently; Table 3.3 presents additional measures available in 
NELS but not in NAEP; and Table 4.4 shows the variables that are unique to the NAEP. 

Test Score Measures 

The achievement measures are the bottom decile and quartile of the 
standardized mathematics and reading tests in NAEP and NELS. The 1990 NAEP 



DRAFT 

Identifying Students at Risk 



21 



Berends & Koretz 



reading test aimed at assessing different modes of comprehension for different types of 
texts (NAEP, 1989; Foertsch, 1992). The mathematics test in the 1990 NAEP aimed at 
assessing a variety of students' mathematical abilities such as conceptual 
understanding, procedural competencies, and problems solving abilities (NAEP, 1988; 
Mullis et al., 1991). The 1990 mathematics assessment asked questions in various 
content areas such as numbers and operations; measurement, geometry; data analysis, 
statistics, and probability; and algebra and functions. Reading performance was 
assessed in different reading domains such as reading a literary text, an informational 
document, and instructions to carry out a task. For a majority of the questions, students 
were required to give written answers. The reading test assumed that reading 
comprehension varies from informational (locating, comparing, and evaluating skills) to 
literary (constructing, examining, and extending meaning). 

NELS also tested students in the areas of mathematics and reading (see Rock 
and Pollack, 1991). The mathematics test lasted longer, thirty minutes, and contained 
forty items required students to make quantitative comparisons and to provide answers 
to word problems, diagrams, and calculations. The reading test consisted of twenty-one 
multiple choice items that measured student comprehension and interpretation of five 
short passages that varied in length from one paragraph to a half-page. The eighth 
graders were given twenty-one minutes to complete this test. We relied on the test 
scores that corrected for guessing provided by the National Center for Education 
Statistics (NCES). 
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Table 3.1 

Variable Definitions Consistent Across Both NAEP & NELS 



Variable Name 


Definition 


Test Score Measures 


Bottom Mathematics Decile 
Bottom Mathematics Quartile 
Bottom Reading Decile 
Bottom Reading Quatile 


Bottom Decile of Weighted, standardized Z- 
score 

Bottom Quartile of Weighted, standardized 
Z-score 

Bottom Decile of Weighted, standardized Z- 
score 

Bottom Quartile of Weighted, standardized 
Z-score 


Demographic Characteristics 


Black 


= 1, if student reports being black 
- 0, otherwise 


Hispanic 


= 1, if student reports being Hispanic 
- 0, otherwise 


White 


= 1, if student reports being white 
= 0, otherwise (reference group) 


Other 


= 1, if student reports being Asian or 
American Indian, etc. 

= 0, otherwise 


Female 


= 1, if student reports being female 
= 0 otherwise 


Family Characteristics 



Mother's Education 

Less than high school = 1, if education < high school graduate 

= 0, otherwise 



High school graduate = 1, if education = high school graduate 

= 0 otherwise (reference group) 

Some college = 1, if education = some college 

= 0 otherwise 

College graduate = 1, if education = college graduate 

= 0, otherwise 

Father's Education 

Less than high school = 1, if education < high school graduate 

= 0, otherwise 



High school graduate = 1, if education = high school graduate 

= 0 otherwise (reference group) 

Some college = 1, if education = some college 

= 0, otherwise 

College graduate = 1, if education = college graduate 

- 0, otherwise 
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Table 3.1 (Continued) 



Variable Name 


Definition 


Family Characteristics (continued) 


Single Mother 


= 1, if student lives in a single parent 
household, headed by mother 
= 0, otherwise 


Mother's Labor Force Participation 


= 1, if parent reports mother works full or 
part time 
= 0, otherwise 


Academic Record 


Homework 


Hours of homework per week 


School Characteristics 


School Sector 
Catholic 

non-Catholic Private 
Public 

School Mean Parent Education 

Percent Black 
Percent Hispanic 
Percent Free Lunch 
School Mean Homework 


= 1, school sector = Catholic 
= 0, otherwise 

= 1, school sector = private, non-Catholic 
= 0, otherwise 
= 1, school sector = public 
= 0, otherwise 

Aggregated from higher of two parent- 
reported education levels 
From school questionnaire 
From school questionnaire 
From school questionnaire 
Aggregated from student reported 
homework 


Community Characteristics 



Locale Dummies 
Rural 

Urban 

Suburban 



= 1, if school located in rural area 
= 0, otherwise 

= 1, if school located in urban area 
= 0, otherwise 

= 1, if school located in suburban area 
= 0, otherwise (reference group) 



Re gion Dummies 
Northeast 

North Central 

West 

South 



= 1, if student lives in Northeast region 
= 0, otherwise 

= 1, if student lives in North Central region 
= 0, otherwise 

= 1, if student lives in the West 
= 0, otherwise 

= 1, if student lives in the South 
= 0, otherwise (reference group) 
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Table 3.2 

Variable Definitions That Differ Across Both NAEP & NELS 



Variable Name 


Definition 


Language Use Characteristics 


NAEP 




Never 


= 1, if student reports never using language 
other than English (reference group) 

= 0, otherwise 


Sometimes 


= 1, if student reports sometimes using 
language other than English 
= 0, otherwise 


Always 


= 1, if student reports always using 
language other than English 
= 0, otherwise 


NELS 




Usually speak English 


= 1, if student reports usually speaking 
English at home 
= 0, otherwise (reference group) 


Usually speak Spanish 


= 1, if student reports usually speaking 
Spanish at home 
= 0, otherwise (reference group) 



Usually speak other language = 1, if student reports usually speaking 

language other than English or Spanish at 
home 

= 0, otherwise (reference group) 
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Table 3.3 

Additional Variables in NELS, not in NAEP 



Variable Name 


Definition 


Family Characteristics 


Family Income 


1987 dollars (0000s) 


Number of Siblings 


Total number of siblings, aged 18 or 
younger 


Mother's age at child's birth 
Immierant Status 


Age of mother at birth of eighth grader 


Mother is immigrant 


= 1, if parent reports eighth grader's mother 
is immigrant 
= 0, otherwise 


Student is immigrant 


= 1, if eighth grader is immigrant 
= 0, otherwise 


Family Closure 


Composite of how many of child's friends 
parents know by name (range 0-6) 


Academic Record 


Number of times eighth grader has 
changed schools 


From parent questionnaire 


Ever held back 


= 1, if parent reports eighth grader was ever 
held back in school 




= 0, otherwise 


GPA 


Composite of student-reported grades since 



6th grade in English, mathematics, science, 
and social studies from student 
questionnaire 



School Characteristics 



School Mean Family Income 


Aggregated from family income 1987 
dollars (000s) 


Eighth grade class size 


From school questionnaire 


School closure 


Aggregated from family closure measure 
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Table 3.4 

Additional Variables in NAEP, not in NELS 



Variable Name Definition 

Community Characteristics 

Size and Type of Community 



Extreme rural 


= 1, if school is located in small place and 
extreme rural 
= 0, otherwise 


Low metropolitan 


= 1, if school is located in big, inner dty 
= 0, otherwise 


High metropolitan 


= 1, if school is located in affluent suburbs 
of big city (reference group) 

= 0, otherwise 


Main big city 


= 1, if school is located in big dty and not 
inner dty, affluent suburbs, or rural area 
= 0, otherwise 


Urban fringe 


= 1, if school is located outside city limits, 
but within urbanized area of a big dty 
= 0, otherwise 


Medium dty 


= 1, if school is located outside in a city with 
population between 25,000 and 200,000 
= 0, otherwise 


Small place 


= 1, if school is located outside in a city with 
population less than 25,000 
= 0, otherwise 
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Demographic Characteristics 

Both surveys included items for identifying students' population group, or race- 
ethnicity. We created dummy variables to classify students into non-overlapping 
categories for African American, Hispanic, white, and other (mostly Asian and 
American Indians). Gender was also a dummy variable equal to one if the student was 
female. 

Family Characteristics 

For both mother's and father's education, we used dummy variables to 
represent the categories for less than high school graduate, high school graduate, some 
college, and college graduate or post-graduate degree. High school graduate was the 
reference category in the logistic regression analyses. Even though the original format 
for the NELS parent education measures included several more categories, these were 
collapsed into four to conform to the NAEP. Single mother and mother working (full or 
part time) were also included as dummy variables in NAEP and NELS. 

Language use measures were also available in the two databases, but the 
questions were different on the NAEP and NELS surveys (Table 3.2). NELS asked 
students what language they used most often in their home. Dummy variables were 
created for the categories usually speak English (reference), usually speak Spanish, or 
usually speak some other language. NAEP asked students how frequently they spoke 
a language other than English. Dummy variables were created for never (reference), 
sometimes, and always. 

Additional family measures in NELS, but not in NAEP, included family income, 
number of siblings, mother's age at child's birth, and mother's and student's 
immigration status (see Table 3.3). Family income is expressed in 1987 dollars (000s). 
Other continuous variables are age of mother at birth of eighth grader and number of 
siblings. A more precise measure of mother's age at child's birth would be the age of 
the mother at the birth of her first child, but NELS did not ask the parent responding to 
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the survey this question. The number of siblings used in the analysis includes only 
siblings who were eighteen years old or younger living in the household. 

The parent responding to the NELS survey gave information about whether the 
eighth grader was an immigrant and whether the student's mother was an immigrant. 
Dummy variables were created for both. Finally, family closure is based on parent 
responses to how many of their child's friends they know by name, ranging from zero 
to six. It is intended to measure the social ties, or "social capital," within the family and 
community surrounding the school (for arguments for this measure see Coleman, 1988; 
Schneider and Coleman, 1993). 

Academic Record 

Both NAEP and NELS included information on students' academic record as 
indicated by the hours of homework per week spent on homework (Table 3.1). Even 
though the original item in NAEP asked students the hours of homework they had each 
day, this was recoded to measure homework per week. 

NELS also included additional information on students' academic records (see 
Table 3.3). School mobility is a parent-reported measure of the number of times the 
eighth grader changed schools since entering the first grade. Retention in grade is from 
the parent survey and measures whether or not the eighth grader was ever held back in 
school. Grade point average (GPA) is a composite based on student reports of their 
grades since sixth grade in English, mathematics, science, and social studies. 

School Characteristics 

Several school measures were similar in both NAEP and NELS. We measured 
school sector with dummy variables for Catholic, non-Catholic private, and public 
schools. School composition variables included the school mean parent education, the 
percentage of the student body that is black, the percentage Hispanic, the percentage of 
the students receiving free or reduced lunch, and the average hours of homework 
completed by the students in the school each week. School parent education was 
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aggregated from the higher of the two parents' education levels reported by the student 
in NAEP and the responding parent in NELS, and school homework was based on 
aggregating the student responses in both databases about the hours per week they 
spent on homework. The other school composition measures in NAEP and NELS were 
based on the school administrator survey. 

Further school characteristics available in NELS, but not NAEP, included school 
mean family income, eighth grade class size, and school closure. The school income 
and closure measures were created from aggregating the parent measures within the 
school, and class size was taken from the school administrator survey. 10 

Community Characteristics 

Community characteristics common to both NAEP and NELS included region of 
the country and locale (i.e., urban, rural, or suburban). Dummy variables were created 
for both region and locale; the reference categories were South and rural, respectively. 

NAEP also include a measure for size and type of community (STOC) that is 
based on information from various sources and various levels of aggregation (for details 
see Berends, Koretz, and Lewis, 1994; Johnson and Allen, 1992). For our purposes, we 
created dummy variables for the various categories, including extreme rural, low 
metropolitan, high metropolitan (reference category), main big city, urban fringe, 
medium city, and small place (see Table 3.4). While this measure is frequently used in 
NAEP reports of student scores and is a strong independent predictor of test scores, its 
validity has been questioned (see Berends et al., 1994; Lipmann, 1993). For example, 
because the occupational profiles contained within STOC are based on principal’ reports 
that may not be accurate and because the NAEP sample is partitioned into STOC 



10 Lee and Smith (1993, p. 171) in their analyses of NELS for a different purpose point out that 
because of the various configurations of middle-grade schools e.g., K-8, K-12, 6-8, 7-9, etc.) 
eighth grade enrollment is useful measure of size. They also report that the correlation between 
eighth grade enrollment and school size is .83. 
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categories that are inconsistent with other representative data, substantive conclusions 
based on the STOC variable are problematic. 

APPROACH 

Determining the adequacy of NAEP's family, school and community measures 
depends on the purposes for which they are used. In the present analyses, we 
examined two purposes: (1) describing low-achieving students according to this 
group's family, school, and community characteristics and (2) predicting the likelihood 
that individual students with certain family, school, and community characteristics will 
score poorly on achievement tests. Our analysis of the latter purpose differs from the 
majority of existing research that examines the independent relationships of social 
context to mean achievement levels rather than low achievement levels (e.g., bottom 
quartile or decile). 

One set of analyses compared the bottom decile and quartile on the test-score 
distribution to the eighth-grade population as a whole in terms of simple univariate 
statistics — that is, one variable at a time. In addition, we estimated the correlations 
among several social context measures at the individual level and school level. 

Logistic regression techniques were used to estimate the independent 
relationships between individual's low achievement levels and social context 
characteristics, holding constant a variety of other social context variables. These models 
estimate the probability that a student with given characteristics will fall into the bottom 
quartile or decile. By controlling for other variables, one can estimate the adjusted 
probability of low achievement for a specific variable of interest. For example, one can 
estimate the probabilities that black and white students score into the bottom quartile, 
accounting for other differences in social context. 11 

11 The logistic regression (logit) model is the appropriate method when the dependent variable 
is dichotomous, since it restrict the value of the predicted probability to range between zero and 
one. The model relates the bottom mathematics quartile of the i ,h individual, Y, to a vector of 
social context characteristics for that individual, x,. The assumed relationship is: 
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In the logistic regressions, we included missing value dummies for all cases with 
missing data on the individual and family variables. In each case, the missing value 
was replaced with the weighted mean for continuous measures and the weighted mode 
for categorical variables. Because there were so few students missing information on 
school and community characteristics (less than one percent of the sample), these cases 
were deleted from the analyses. In the case of missing parental education in NELS, we 
first substituted the education of the other parent, then the student report of parent 
education, and then the weighted mean of the variable itself for the remaining missing 
values. Replacing missing values for individual variables preserved the large sample 
sizes of the databases and prevented us from throwing out cases that had important 
non-missing information. In addition, explicitly controlling for the missing data allows 
us to see whether these individuals differ from the non-missing cases in some 
systematic way. A sensitivity analysis was performed utilizing only the cases with all 
data present, and results were substantially the same as those obtained by including 
missing data variables. 



Y, = P (Xi) + El, 
P(Xi) = P[Y, = 1 I Xi] 
= 1 / 1 + 



where P(xi) 

Xij 



P; 

Po 

We present 



= probability of scoring in the bottom quartile for specific eighth grader i, 

= values of the explanatory social context variable; for individual i, 

= estimated coefficient for the Xj, and 
= estimated constant term. 

the maximum likelihood estimates in Appendix B and the probabilities in Chapter 5. 
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4. 



DISTINGUISHING CHARACTERISTICS OF 



LOW-ACHIEVING STUDENTS 

Simple univariate descriptions of low-achieving students reveal that several 
characteristics presently in the NAEP are useful for the purpose of differentiating low 
achievers as a group from the eighth grade student population. However, analyses of 
NELS show that the NAEP currently lacks several measures important for this purpose. 

Before reporting the detailed results, it is important to note that comparisons 
within databases — whether between mathematics and reading tests or between the 
bottom quartile and decile — were strikingly similar. For example, the percentages of 
students in the bottom quartiles belonging to each population group (black, Hispanic, 
and white) were similar to the corresponding percentages in the bottom deciles. The 
similarity of the bottom decile and quartile is surprising; if scores are approximately 
normally distributed and have roughly similar variances in each group, one would find 
that lower-scoring groups are more over-represented in the bottom decile them in the 
bottom quartile. Because of these similarities across cut-points and subject areas, we 
will often discuss only one set of results rather than all four. A complete set of results 
appears in Appendix A. 

Table 4.1 provides a summary of the characteristics that substantially 
differentiate between the group of low-achieving students and the eighth grade 
population. "Substantially" here is arbitrarily defined as more them a five point 
difference between the percentage of students in the bottom quartile and the percentage 
of the entire population consisting of students from a given group. The table contrasts 
the bottom quartile to the population as a whole in mathematics, based, on analysis of 
NELS; variables not presently available in the NAEP are shaded. Other contrasts 
between low-achieving groups (e.g., bottom decile vs. population), between tests within 
databases (mathematics vs. reading), and between databases (NAEP vs. NELS) are quite 
similar, so Table 4.1 provides an illustrative summary of the more detailed analyses that 
follow. 
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Table 4.1 

Percentage of Mathematics Achievement Groups in NELS with 
Certain Social Context Characteristics 





Bottom 








Quartile 


Population 


Difference 


Demographic Characteristics 


Black 


26% 


13% 


-13% 


Hispanic 


15% 


9% 


-6% 


White 


54% 


73% 


19% 


Family Characteristics 


Mother's Education High School or Less 


58% 


43% 


-15% 


Father's Education High School or Less 


61% 


43% 


-18% 


Mother is Single Parent 


22% 


16% 


-6% 


Family Income < ' ?!• - 




20% 


- 17 % 


More than Two Siblings 


;:: : 42% ; ■ 




:~8% 


iMol^er’s Age at C|?iM’s 




:13% ' ' 


>- 7 % - 


Family Closure £ 2-bf Ch^ 


' ^ , ; 38%^ 


6% j 


-22% 


Academic Record Characteristics 




^39%f • 


i -32%, 


mim§§ 


Ever Held Back 


41% 1 




WlmSm 


GPA < C Average • 


;;^|f§i%3i: 


' 16% 


1 45 % 


School Characteristics 


School Percent Free Lunch > 50% 


25% 


16% 


-9% 


School Percent Minority > 25% 


56% 


37% 


-19% 


School Mean Parent Education 13 Years or Less 


37% 


23% 


-14% 


School Mean Homework < 5 hours/week 


45% 


37% 


-8% 


School Mean Income < $25,000 




20 % -v 


: ; - - 16 % 


School Mean Closure S 2 


llpill 




^12% : 


Community Characteristics 




- 




South 


44% 


36% 


-8% 



Notes : Difference is equal to the percentage in the population minus the percentage in 
the bottom quartile. The highlighted rows indicate variables in NELS but not in NAEP. 
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In general, a wide array of social context measures for population group, family, 
academic record, school, and community distinguish low achievers from the 
population. For example, while a majority of low achievers are white, whites are under- 
represented in the low-achieving group when compared to the population. Fifty-four 
percent of the group scoring in the bottom mathematics quartile is white, but 73 percent 
of the NELS population is white. In addition, minorities are over-represented in the 
low-achieving group, especially black students. 

Family characteristics that differentiate students at risk of low achievement 
when compared to the population include low levels of mother's and father's education, 
single mother households, low family income, large families, mothers who were young 
when they gave birth to the eighth grader, and low levels of family closure (e.g., parents 
know few of their child's friends by name). Family income, family size, age of mother 
at birth of child, and family closure — while important — are absent from NAEP. 

Students' academic records are important for comparisons between the group of 
low achievers and the population. Students who changed schools frequently prior to 
eighth grade, who were held back, and who had below-average grade point averages 
were over-represented in the low-achieving group when compared to the population. 
None of these measures is in NAEP. 

In addition to these individual characteristics, school characteristics are useful 
for describing the low-achieving groups of students. When compared to the 
population, students in schools with greater percentages of minorities, low 
socioeconomic status, and lower levels of homework and closure are disproportionately 
represented in the low-achieving group. For example, just over one-third of the 
population attends schools in which more than 25 percent of students are minority, but 
56 percent of students in the bottom quartile attend such schools. In addition, 36 
percent of low-achieving students are in schools with average family incomes less than 
$25,000 (1987 dollars), as compared to 20 percent of the population, a 16 point 
difference. Other school composition characteristics — such as a greater percentage of 
students on free lunch, low school mean parent education, low levels of homework, and 
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low school closure (e.g., on average parents in the school do not know their children's 
friends) — substantially distinguish between the low-achieving group and the 
population. Two of these school-composition variables, school mean income and 
closure, are presently lacking in the NAEP. 

While both NAEP and NELS lack adequate measures of community 
characteristics, both include measures of region, which is useful for distinguishing the 
low-achieving groups from the population as a whole. Low-achieving students tend to 
be over-represented in southern states compared to the population. Thirty-six percent 
of the NELS population is located in the South, for example, but 44 percent of students 
in the bottom quartile were in the South. 

Measures for urban locale (e.g., urban, rural, and suburban areas) provide 
inconsistent contrasts between groups both within and between databases, so they are 
not useful for describing students at risk. In NAEP, which contains a measure for size 
and type of community (STOC), low-achieving students are under-represented in the 
affluent suburbs (i.e., "high metropolitan" category of STOC) compared to the 
population. However, the validity of STOC is questionable, so using it to describe 
students at risk is ill-advised. In short, because both databases lack good measures of 
neighborhood and community characteristics, it is difficult to know whether these 
social contexts are important for describing students at risk of low achievement. Future 
research should consider this issue. 

Other characteristics that were expected to distinguish low-achieving eighth 
graders from the population, but do not, include mother's labor force participation, 
students' individual homework levels, and school sector (private vs. public). In 
addition, low-achieving students did not strongly differ from the population according 
to whether English was the primary language spoken in the home. NELS also included 
questions that asked whether the parent and the student were immigrants. Neither of 
these measures differentiated substantially between low achievers and the population 
although some minor differences do appear. Finally, while gender differentiates low 
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achieving groups from the population in reading (i.e., females are under-represented in 
low-achieving groups), there are not gender differences in mathematics. 

The following sections provide in detail the results from our analysis of NAEP 
and NELS. Various sets of social context measures are discussed in turn, including 
measures for demographic, family, language use and immigration status, academic, 
school, and community characteristics. Rather than present all of the comparisons 
between subject areas and between databases, we highlight those findings that are 
informative and note differences that occur. 



Demographic Characteristics 

Although the majority of students who are at risk of low achievement are white 
(because most of the student population is white), whites are under-represented in the 
group of low-achieving students. In NAEP, about 60 percent of students scoring in the 
bottom reading quartile are white (see Figure 4.1), and 46 percent of those in the bottom 
mathematics quartile are white (see Figure 4.2), while about 70 percent of the NAEP 
sample is white. 
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Figure 4.1— Percentage of Reading Achievement Groups in NAEP with 
Certain Demographic Characteristics 
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Figure 4.2— Percentage of Mathematics Achievement Groups in NAEP with 
Certain Demographic Characteristics 



Conversely, minorities are over-represented in the low-achieving groups. For 
instance, Table 4.1 shows that in reading 22 percent of the low-achieving group is black 
and about 17 percent is Hispanic compared to the NAEP reading population, which is 
15 percent black and 10 percent Hispanic. These patterns are even more striking in 
mathematics (see Figure 4.2). 

Females are under-represented among low achievers in reading but not in 
mathematics. Whereas the NAEP sample is 50 percent female, the low-achieving group 
in the bottom reading quartile is 36 percent female. Similar patterns for gender also 
appear in NELS (see Appendix A.2). 

The comparisons between the bottom quartile and decile are strijkingly similar. 
For most of the demographic characteristics, the differences between the bottom 
quartile and decile do not differ by more than a few percentage points. For example, 
whether considering the bottom quartile or decile, 22 percent of the low-achieving 
group is black, 16 - 18 percent is Hispanic, and 58 - 59 percent is white. Furthermore, 
Figure 4.2 shows that being female does not differentiate those who score in the bottom 
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decile from those in either the bottom quartile or the entire population of eighth 
graders. There are a few exceptions, however, such as in NAEP mathematics, where 33 
percent of students in the bottom quartile are black and 41 percent of those in the 
bottom decile are black. Another exception is that while gender does not differentiate 
between those in the bottom decile and quartile in most instances, it does in NAEP 
mathematics, where females comprise a smaller share of the bottom decile than of the 
bottom NAEP quartile (see Figure 4.1). 

Family Characteristics 

A variety of family characteristics differentiate students in the bottom quartile 
and decile from the population as a whole. Figure 4.3 depicts family characteristics that 
are currently in the NAEP (e.g., parents' education, single mother household, mother's 
labor force participation). 



Insert Figure 4.3 

Parents' educational attainment levels quite strongly differentiate students who 
are low achievers from the eighth-grade student population but do not strongly 
differentiate students scoring in the bottom decile from those scoring in the bottom 
quartile. In NELS, 58 percent of the students in the bottom quartile in mathematics 
have mothers who did not graduate from high school, compared to 43 percent in the 
population. Moreover, 43 percent of the students in the NELS population have fathers 
without a high school diploma compared to 61 percent of those scoring.in the bottom 
mathematics quartile. 

This same general pattern of overrepresentation in the low-achieving groups of 
students whose parents lack a high school diploma is apparent in NAEP, even though 
NAEP's measures of parental education (student reports) differ somewhat from the 
NELS measures (from parental reports) (see Appendix A.l and A.2). NAEP reports a 
higher percentage of mothers without a high school diploma. Fifty percent of the 
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students in the NAEP reading population report that their mothers are not high school 
graduates, but 43 percent of the students in NELS have mothers with less than a high 
school diploma. 

Students in the low-achieving groups are also more likely than students in the 
population as a whole to be children of a single mother, but the differences are much 
smaller than those for parent education. The largest differences appear in mathematics 
(Figure 4.3). Sixteen percent of the students in the NELS population have mothers who 
are single parents; 22 percent of the low-achieving groups have such mothers. 

The measures for mothers labor force participation — full or part time — do not 
strongly differentiate low-achieving students from the population, and the differences 
only appear in some of our comparisons. Students whose mothers work part time are 
slightly under-represented in the low-achieving groups on the NELS mathematics test 
compared to the population — 18 percent of the NELS population has mothers who 
work part time, but only about 14 percent of students in both the bottom mathematics 
quartile — but in NAEP, there is practically no difference between the low-achieving 
groups and the population in terms of mother's work status outside the home. 

A number of family characteristics not presently in the NAEP differentiate 
students at risk of low achievement from the eighth-grade population. These include 
family income, family size, mother's age at birth of the child, and family closure. 

Income strongly differentiates the low-achieving groups from the population. Figure 
4.4 shows that 37 percent of students who score in the bottom mathematics quartile are 
in families that earn less than $15,000 compared to 20 percent of the population that are 
in such families. 12 While 34 percent of the students in NELS have more .than two 
siblings, over 40 percent of low those in the low achieving groups live in large families. 
Moreover, 13 percent of the NELS students have mothers who were twenty years old or 
younger when the eighth grader was bom, but 20 percent of the students in the bottom 

12 The $15,000 cutoff that we use is the poverty rate for a family of six people in 1987 dollars. 
About 20 percent of the youth population in 1987 were living in poverty (see U.S. Bureau of the 
Census, 1989; U.S. Department of Education, 1993). 
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mathematics quartile were bom to young mothers. Finally, between 38 - 41 percent of 
students in the low-scoring groups are in families where the parent knows only two or 
fewer of the child's friends by name (i.e., low family closure), in contrast to twenty-six 
percent in the eighth grade population. 
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Figure 4.4-- Percentage of Mathematics Achievement Groups in NELS with 
Certain Family Characteristics Not in NAEP 



Language Use and Immigrant Status Characteristics 

NAEP and NELS measured language use differently. In NAEP, students were 
asked whether they used another language other than English; their responses were 
limited to "never," "sometimes," or "always." NELS asked several questions about 
language use, but we relied on the item that asked whether the students' usually spoke 
English, Spanish, or some other language other than English in their homes. 

In NAEP, the language use measure does not differentiate low-achieving 
students from the population in reading but did slightly in mathematics. Figure 4.5 
shows that 70 percent of the NAEP mathematics population never speaks a language 
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other than English, whereas 66 percent of students in the bottom mathematics quartile 
and 63 percent in the bottom decile do so. The reverse pattern is seen among those who 
report "always" use a non-English language. While 7 percent of the population report 
always speaking a language other than English, 11 to 12 percent of low achievers report 
doing so. 
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Figure 4.5~Percentage of Mathematics Achievement Groups in NAEP with 
Certain Language Characteristics 



Similar to NAEP, the language-use measure in NELS differentiates low- 
achieving students, but the differences between groups are small. Figure 4.6 shows that 
5 percent of the NELS population usually speaks Spanish at home, whereas about 10 
percent of those scoring in the bottom reading quartile usually speak Spanish. A 
similar pattern appears for low-achieving groups on the NELS mathematics test. 
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Figure 4.6— Percentage of Mathematics Achievement Groups in NELS with 
Certain Language and Immigration Status Characteristics 



Immigrant status, whether of the eighth grader or of the eighth grader's mother, 
does not differentiate between low-achieving students and the general population. Five 
percent of the students in NELS were bom outside the United States, and 11 percent of 
the eighth graders mothers were bom in another country. Figure 4.6 shows that low- 
achieving groups in reading differ little from the population when considering the 
immigrant status measures. We found similar results in mathematics. 

Academic Record 

Eighth grader's academic records (e.g., school mobility, grade retention, grade 
point average, and homework) differentiate groups who score poorly on mathematics 
or reading from the population. Mobility and homework showed only moderate 
differences, while retention and GPA differentiated more strongly. Unfortunately, of 
these four variables, only self-reported homework is available in NAEP. Figure 4.7 
shows that while 32 percent of the NELS population changed schools more than once 




DRAFT 43 Berends & Koretz 

Identifying Students at Risk 



since the first grade, 38 to 39 percent of students in the low-achieving groups changed 
schools. 
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Figure 4.7—Percentage of Reading Achievement Groups in NELS with Academic 

Record Characteristics 



The measure in NELS for grade retention substantially differentiates students at 
risk of low achievement from the population. For example, 42 percent of students 
O scoring in the bottom reading decile and 38 percent in the bottom quartile were held 

back during their elementary schooling years, compared to 19 percent in the eighth- 
grade population. The differences between low-achieving mathematics students and 
^ the population are even more striking (see Appendix A.2). 

Unsurprisingly, students' self-reports of their cumulative grade point averages 
across subjects since the sixth grade substantially differentiate low achievers from the 
population of eighth grade students. For instance, while 16 percent of the NELS 

» 

students reported a C average or below, about 30 percent of those in the bottom reading 
quartile and about 35 percent in the bottom decile report a GPA of C or below. 

The amount of homework per week the student reports distinguishes only 
® modestly between low-achieving students and the population. For example. Figure 4.7 
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reveals that 10 percent of NELS students completed 1.5 hours or less of homework per 
week, whereas 15 percent of those scoring in the bottom reading quartile and about 16 
percent of those in the bottom decile completed this amount. A homework measure 
was also available in NAEP, and the patterns are even more striking than those in 
NELS, especially on the NAEP reading test (see Appendix A.l). 

School Characteristics 

Several school characteristics available in the NAEP are useful for distinguishing 
low-achieving students from the eighth-grade population as a whole. Some of these are 
aggregate values of individual characteristics noted above (e.g., parent education and 
homework). Low-achieving students are disproportionately in schools characterized as 
having a significant percentage of students who receive free lunch, who are minority, 
who have parents with low education levels, and who complete small amounts of 
homework. Low-achieving students are under-represented in private schools. 13 

Figure 4.8 provides an illustration of the school characteristics in NAEP that 
differentiate students at risk of low achievement from the eighth-grade population. 
While 11 percent of student in the NAEP population attend private schools, 5 percent of 
those scoring in the bottom quartile and 4 percent in the bottom decile are in private 
schools. Students in schools where more than 50 percent of the student body receives 
free or reduced lunch or who attend schools that are more than 25 percent minority are 
also over-represented in the bottom quartile and decile. 14 (Although free lunch is often 
used as a proxy for school poverty, comparisons with school income levels (and 



13 For these analyses, both Catholic and other private schools were considered private. 

14 The threshold of more than 50 percent of the students receiving free or reduced lunch is 
based on recent federal legislation — The Improving America's Schools Act of 1994 -- that 
enables Title I schools to develop school-wide programs if half of its students are poor. 
Previously, 75 percent of a school's student population had to be poor to develop school-wide 
programs with Title I funds. 
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analyses in Chapter 5) reveal that these variables do not identify the same groups of 
students.) 



Insert Figure 4.8 

The average parent education of students in the school also differentiates 
between low achievers and the NAEP population. In mathematics, 33 percent of the 
students in the NAEP are in schools where the average parent education level is 
thirteen years or less (i.e., one year beyond high school graduation), but 49 percent of 
low achievers are in such schools. 

Finally, the average level of homework assigned by the school differentiates 
students who score poorly on academic achievement tests from the general population. 
For example, 20 percent of the NAEP population, compared to about one-fourth of the 
students in the bottom quartile and decile, attend schools where the average amount of 
homework assigned in the school is less than 5 hours per week. 

There are some exceptions to these patterns when considering reading in NAEP 
and the NELS tests. In both NAEP and NELS school measures for free lunch, minority 
composition, and parent education more strongly differentiate low-achieving groups 
from the population in mathematics than reading. Moreover, these measures within 
NAEP also distinguish the bottom decile from the bottom quartile in mathematics, but 
not in reading (see Appendix A.1 and A.2). 

Two measures of school characteristics available in NELS that are absent in the 
NAEP — school mean income and school closure — strongly differentiate between low 
achievers and the population as a whole. Students in the low-achieving groups are 
over-represented in schools that have low mean incomes and have low levels of school 
closure (e.g., connections among families). Figure 4.9 shows that 36 percent of students 
scoring in the bottom quartile and 39 percent of those in the bottom decile attend 
schools where the average family income level is less than $25,000, compared to 20 
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Figure 4.8~Percentage of Mathematics Achievement Groups in NAEP with Certain School Characteristics 





percent of the NELS population in such schools. 15 In addition, while one in five 
students in the population attend schools that have low levels of school closure, about 
one-third of the low-scoring students are in schools with low levels of closure. 
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Figure 4.9— Percentage of Mathematics Achievement Groups in NELS with 
School Characteristics Not in NAEP 



Community Characteristics 

Very few of the community measures we analyzed provided consistent contrasts 
between students at risk of low achievement and the general population. Because 
NAEP and NELS yield inconsistent findings for urban locale, relying on the urbanicity 
measure to report comparisons between low achievers and the population is ill-advised. 
The NAEP size and type of community measure differentiates groups, but the validity 
of STOC has been criticized, so using it to report comparisons is questionable. Some 
interesting contrasts comparisons appear for region (e.g., low achievers are over- 

15 The school income cutoff of $25,000 is 70% of the school median income. Even though this is 
an arbitrary threshold, about 20% of the sample reports being in such schools which is similar to 
the percentage of the 1987 youth population living in households below the poverty line (U.S. 
Census Bureau, 1989; U.S. Department of Education, 1993). 
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represented in the South), but generally high quality measures of community and 
neighborhood characteristics are lacking in both NAEP and NELS. 

In sum, our univariate comparisons between low-achieving groups and the 
eighth grade population suggest that while NAEP presently contains several 
demographic, family, and school measures useful for this purpose, but it could be 
substantially improved by including additional measures like those found in NELS 
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5. ASSESSING NAEPS ADEQUACY FOR 
PREDICTING LOW ACHIEVEMENT 



The univariate analyses above showed that when examined individually, a wide 
array of social context measures — some available in the NAEP, some not — 
differentiate students with low test scores from the population. However, many of 
these social context measures are strongly related to each other, so their univariate 
relationships with low achievement can be quite different from their independent 
effects on achievement when other aspects of social context are held constant. 

Therefore, this chapter examines the independent effects of social context 
measures on the probability that a student will score in the bottom quartile or decile, 
after other social context measures are taken into account. Specifically, it addresses 
several questions: First, what social context measures are independently related to low 
achievement apart from their relationships with other factors? For example, are 
students from poor families more likely to score in the bottom quartile them non-poor 
students after accounting for other differences in social context? Second, which 
measures presently in the NAEP have the most important independent relationships 
with the probability of having low achievement scores? Third, what measures should 
be added to the NAEP to make it more useful in this regard? Finally, to what extent can 
measures currently in the NAEP serve as proxies for those that are omitted? 



Logistic regression was used to assess the independent relationships between 
social context measures and the probability of scoring in the bottom quartile or decile in 
reading or mathematics. (Logistic regression was used because ordinary least squares 
regression can yield biased estimates when an outcome variable is dichotomous and the 
probability of being in one of the categories is small.) Logistic regression estimates the 
probability that students with specified combination of social context characteristics will 
score poorly. By definition, the overall (unadjusted) probability of scoring in the bottom 
quartile is .25; the estimated probability for students with any given combination of 
social context characteristics could in theory range from 0 (no such students are in the 
bottom quartile) to 1 (all of them are). If the values for other social context measures are 
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set to the grand mean, these models estimate the probability that a student who has a 
given characteristic but is in other respects average would score in the bottom quartile 
or decile. For example, a student with a household income of $15,000 has a probability 
of .34 of scoring in the bottom quartile, as compared to a probability of .25 for the 
population as a whole. However, if a large enough number of other social context 
variables (for example, parental education) are held constant at the grand mean, 
students with a household income of $15,000 are a bit less likely than average 
(probability = .21) to score in the bottom quartile. 

A word of caution is needed about interpreting the "independent effects" 
estimated in these models. Many of the social context measures analyzed here are 
substantially collinear (i.e., related to each other). Impoverished parents are more likely 
than others to be poorly educated, for example. If enough collinear variables are 
entered into a model, the independent "effects" of many of them will appear small. For 
instance, in the example given above, when enough other social context variables are 
taken into account, moderately low income no longer predicts a higher-than-average 
probability of low achievement. In that same model, the mother having less than a high 
school education also shows no independent association -- holding other variables 
constant — with the probability of low achievement. This finding does not mean that 
neither income nor parental education "really" affects student achievement. These 
models do not show which of the collinear variables actually causes differences in 
achievement, and either maternal education, income, or both could in fact be powerful 
influences on student achievement. The model simply says that once one has taken a 
large enough number of these related variables into account, information about one 
additional measure does not add further to one's ability to predict low achievement. 

The models reported here include individual-level variables (student and family 
characteristics), school characteristics, and community characteristics. Some of the 
school characteristics are aggregate values of individual variables (i.e., parent education, 
homework, income, and closure); the other school measures are based on school 
reports. All of the analyses reported here use NELS, although some of them (the 
"NAEP-like" models) are restricted to variables included in NAEP. This allows us to 
examine both the importance of variables in NAEP and the impact of NAEP's omission 
of some variables without confounding these issues with other, irrelevant differences 
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between the databases. Since the logistic regression results are similar in mathematics 
and reading and in the bottom quartile and decile, only the results for the bottom 
quartile in mathematics are presented here. 

The analysis involved several steps. First, to examine the independent 
associations between low test scores and social context measures, we present three sets 
of probabilities. We first present the unadjusted probability of scoring poorly for each 
social context measure by itself -- for example, the probability that a student who comes 
from a poor family would score in the bottom quartile without accounting for any other 
differences in social context. (These are the probabilities discussed in the previous 
chapter.) We also show the adjusted probabilities of scoring in the bottom mathematics 
quartile that control for a number of other social context measures available in NAEP. 
We refer to this specification as the NAEP-like model. In addition, we present 
probabilities that adjust for a larger set of social context measures, some of which are 
not currently in the NAEP (e.g., the full NELS model, which includes a substantial 
number of variables available in NELS but not NAEP). These three sets of probabilities 
allow us to assess what variables presently in the NAEP are most important for 
predicting low mathematics scores and what important variables are missing from 
NAEP. 

Second, to further assess whether specific measures would improve the NAEP's 
adequacy to predict low achievement, we add measures not currently in the NAEP one 
at a time to the NAEP-like model. For example, we add family income to examine 
whether income has a significant effect on the probability of low achievement over and 
above those variables already in the model and available in NAEP. This helps 
determine the extent to which variables included in NAEP serve as proxies for those 
that are omitted. For example, if income adds little to the prediction that a student will 
score in the bottom quartile, it can be argued that the variables included in NAEP are a 
sufficient proxy for income/or that particular purpose. 

Third, a number of social context variables were regressed on other such 
variables; for example, regression was used to estimate the extent to which family 
income is predicted by the other social context variables in NAEP. These regressions 
help clarify both the problem of collinearity and the utility of NAEP's variables as 
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proxies for other, omitted variables. Bivariate correlations were also examined to 
further investigate the issue of proxy measures (see Appendix B). 

SUMMARY OF RESULTS 

As expected, both of our models (the NAEP-like model and the full NELS 
model) showed that when one holds constant a large number of social context variables, 
numerous variables that showed strong univariate relationships with low achievement 
do not show a large independent association with low achievement. For instance, 
measures for single parent households, school minority composition, and percentage in 
the school on free lunch differentiated low-achieving groups from the population in the 
previous chapter, but they were not independently associated with low achievement 
scores in the analyses that follow. While some of these measures remain statistically 
significant in our multivariate models, the effects are trivial. For example, students in 
primarily black schools are barely more likely to score poorly than students in primarily 
white schools once a variety of other variables (including the students' own population 
group) are taken into account. This contrasts the univariate descriptions of the last 
chapter that show sharp differences between the percentage of low achievers and the 
population who attend more racially and ethnically diverse schools. Furthermore, 
language use, mother's labor force participation, school sector, and urban locale, which 
do not strongly differentiate low achieving groups from the population (see the 
previous chapter) are also not related to low test scores when controlling for other 
factors. 

However, several demographic, family, and school characteristics presently in 
the NAEP continue to be related to low achievement scores, even after holding constant 
a large number of other social context variables. The demographic and family 
characteristics that have the strongest independent association with low achievement 
are population group membership and parents' educational attainment, which show 
substantial relationships even after other family, school, and community differences 
have been taken into account. Black students are much more likely to score poorly than 
Hispanics and especially whites from similar social contexts. In addition, while 
students whose mothers or fathers only graduated high school have probabilities of low 
achievement almost equal to the population, eighth graders who have college-educated 
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parents are much less likely to score in the bottom mathematics quartile even after other 
factors have been taken into account. 

School characteristics measured in NAEP also show substantial independent 
associations with the probability of being a low-achiever even after taking other 
individual and school characteristics into account. There are striking differences 
between students in schools that have highly educated parents and those that do not. 
Students in schools where the average level of parent educational attainment is less 
than high school graduation have about a fifty percent chance of low mathematics 
achievement (p = .50). Students in schools with highly educated parents have about a 
one in ten chance of scoring poorly (p = .10). In addition, students in schools with 
greater amounts of homework assigned are less likely to score in the bottom quartile. 

Some of the social context variables that are included in NELS but not NAEP 
have substantial independent relationships with the probability of being a low achiever. 
The most important characteristics absent from the NAEP include measures for grade 
retention and immigration status. The adjusted probability of scoring in the bottom 
mathematics quartile is .42 for students who report being held back prior to eighth 
grade — seventeen points above the population probability. Immigration status, both of 
the student and the eighth grader's mother, are strongly related to test scores, but in the 
opposite direction of what is often assumed. Controlling for other differences in social 
context, students who are immigrants (or whose mothers were born outside the U.S.) 
are much less likely to score in the bottom mathematics quartile. The adjusted 
probability of low achievement for immigration status is .15. 

In addition, although the NAEP-like and full NELS models are similar in many 
respects, some of the estimated relationships differ in important ways. This may mean 
that the omission of some social context variables in NAEP leads to biased estimates of 
their effects. For example, when estimating the independent association between school 
mean parent education and low test scores, the model that adjusts only for variables in 
NAEP estimates a higher probability of low achievement, especially for students 
attending schools where parents are less educated, because it does not separate the 
effects of certain social context variables that are related to parental education. 

On the other hand, a number of other characteristics not measured in the NAEP 
either are not statistically related to low scores or have only weak relationships to it. 
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These include family income; family size; mother's age at birth of the eighth grader; 
family closure; school mobility; and school income, closure, and size. For instance, 
there is a only very weak relationship between scoring poorly in mathematics and 
family income after other differences in social context have been taken into account. 

These findings raise the question: how adequate are the social context variables 
in NAEP as proxies for measures of income that NAEP lacks? The answer depends on 
the purpose to which the variables are put. If the goal is only to predict the probability 
that a student will be a low achiever, these results suggest that a large enough number 
of the NAEP social context variables, taken together, are a fairly good proxy for income. 
That is, adding an income variable to the them will not dramatically change the 
predicted probability of scoring in the bottom quartile. 

However, further analyses show that for other purposes -- for example, to 
monitor the achievement of poor students — the variables included in the NAEP do not 
provide an adequate proxy for income. For example, regressing the family income 
measure on the other social context characteristics available in NAEP yields an adjusted 
R 2 of .31. That is, the other social context variables predict only about a third of the 
variance in income. Because family income has a strong unadjusted relationship with 
low test scores and a weak relationship once other factors are controlled, we conclude 
that the portion of family income that matters in predicting the probability of low 
achievement is the third of the variance in income that is related to the other social 
context variables in our full NELS model. 

The sections that follow provide further detail about our analyses. First, we 
present three sets of probabilities, one set for the unadjusted and two sets for those that 
adjust with variables available in NAEP and those that control for a wider set of 
measures. Second, we present the results that show what the net impact would be of 
adding particular social context measures one at a time to those already in NAEP. 

Third, we further explore the issue of adequate proxies in NAEP by showing the 
relationships among the social context measures based on ordinary regressions. 

PREDICTING LOW TEST SCORES WITH SOCIAL CONTEXT MEASURES 

Table 5.1 shows three sets of estimated probabilities. The first column of 
numbers reveals the unadjusted probabilities that do not control for any other social 
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context measures. The second column lists the probabilities that adjust for 
demographic, family, and community measures that are in NAEP, but based on NELS 
data. The probabilities in the third column take into account an even wider set of social 
context measures beyond those available in NAEP. The shaded areas of the table 
highlight those additional variables available in NELS but not in NAEP that were 
included when estimating the probabilities in the third column ( na indicates "not 
available" in NAEP). 

The adjusted probabilities were calculated (from the logistic regression models) 
by setting other social context measures to the grand mean except for the variable of 
interest, so these models estimate the probability that a student with a certain 
characteristic, but otherwise "average," would score in the bottom mathematics 
quartile. While the interpretation of the variables that were dummy coded (e.g., 
population group, parental education, etc.) is straightforward, the interpretation of 
estimates for continuous variables is less so. In Table 5.1, the continuous variables have 
been set to specific values that are noted in the first column; for example, annual income 
was set to $15,000 (the poverty level for a family of six in 1987 dollars), and the values 
in the three columns to the right are the unadjusted and adjusted probabilities that 
students with a family income of $15,000 would score in the bottom quartile. A more 
informative approach, presented later, estimates the adjusted probabilities of low 
achievement for a broad range of values on the continuous variables. 

Before getting to the specific results, it is important to note several general 
patterns in Table 5.1. First, many of the relationships between particular social context 
measures and low test scores, while strong in the univariate comparisons, are 
substantially reduced in the multivariate analyses. Of course, as we previously pointed 
out, the weaker relationships are expected when adjusted for such a large number of 
other factors, and we need to be careful in the interpretation of the probabilities based 
on such adjustments. 

Second, the table shows that those probabilities that adjust for only those 
variables in NAEP and those that adjust for even more measures are strikingly similar. 
While we will note some exceptions in the next section, this similarity between the 
models suggests that for the specific purpose of predicting the probability of low 
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achievement, those variables presently in NAEP, taken together, serve as a reasonably 
adequate proxy for those that are missing. 

Third, consistent with what has been found in the literature on risk, students 
who experience only one risk factor are not necessarily likely to be low achievers (see 
Rutter, 1990; Rolf et al., 1990; Werner and Smith, 1992; Masten, 1994; Grissmer et al., 
1995). For instance, students are not more likely to be low achievers when compared to 
the population if they are "average" except for one of the following characteristics: 
have parents with low educational attainment levels, live in poverty, have a mother 
who is a single parent, are in large families, have parents who don't know their child's 
friends, are primarily Spanish speakers, or in minority schools or low socioeconomic 
status schools. Table 5.1 shows that single measures associated with greater adjusted 
probabilities of low achievement include being black or being held back in school. 
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Table 5.1 



Unadjusted and Adjusted Probabilities of 
Scoring in the Bottom Mathematics Quartile in NELS. 





Unadjusted 

Probabilities 


Adjusted 
Probabilities 
Variables in 
NAEP 


Adjusted 
Probabilities 
Full NELS 
Model 


Population Group Characteristics 


Black 


0.52* 


0.37* 


0.34 


Hispanic 


0.39* 


0.24* 


0.23 


White 


0.18* 


0.18* 


0.17 


Family Characteristics 


Mother's Education High School or Less 


0.46* 


0.28* 


0.24* 


Mother's Education High School Graduate 


0.27* 


0.23* 


0.22* 


Mother's Education Some College 


0.22* 


0.21* 


0.20* 


Mother's Education College Graduate 


0.09* 


0.13* 


0.14* 


Father's Education High School or Less 


0.30* 


0.29* 


0.25* 


Father's Education High School Graduate 


0.45* 


0.24* 


0.22* 


Father's Education Some College 


0.22* 


0.20* 


0.19* 


Father's Education College Graduate 


0.10* 


0.14* 


0.15* 


Mother Is Single Parent 


0.33* 


0.21 


0.18 


Mother Works Full Time 


0.24* 


0.21* 


0.20 


Mother Works Part Time 


0.19* 


0.20* 


0.19 


Family Income ~ $15K 


■I1&34* Hi 


■1 m 


0.21* . 


Number ofSiblings = 3 | M 






Ip0;20 : : !; ■ 


Mother's Age at Child's Birth =20 


§■ 11111 ! 


: na 




High Family Closure <“5) §§|: 


0.15* 


•II . na ih i 


0.18* . 


Language Use Characteristics 


Spanish Usually Spoken at Home 


0.44* 


0.22 


0.23 


Language Other than English Usually Spoken at Home 


0.25 


0.22 


0.23 


Immigration Status Characteristics 


Student Was Bom Outside U.S. § |f| 


Hi 0.26-: 


■:.na 


9x5. 0.15* ti 


Mother Was Bom Outside U.S. 


Il>®.27*v--.r 






Academic Record 




- 




Changed Schools More than Once 


. fO;3S* - 




: : 0.20 !|| 


Ever Held Back 


.0.52* 


&M.mf 


•|,; : :(L4 2*. | 


Homework = 10 Hours/Week 


0.18* 


0.18* 


0.17* 
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Table 5.1 (continued) 



Unadjusted and Adjusted Probabilities of 
Scoring in the Bottom Mathematics Quartile in NELS. 







Adjusted 


Adjusted 






Probabilities Probabilities 




Unadjusted 


Variables in 


Full NELS 




Probabilities 


NAEP 


Model 


School Characteristics 


Catholic School 


0.17* 


0.22 


0.24 


Other Private School 


0.24* 


0.19 


0.20 


School Percent Free Lunch = 50% 


0.36* 


0.22 


0.20 


School Percent Black = 25% 


0.29* 


0.22* 


0.20* 


School Percent Hispanic = 25% 


0.29* 


0.20* 


0.20 


School Mean Parent Education =14 


0.20* 


0.19* 


0.18* 


School Mean Homework = 10 hours/week 


0.12* 


0.18* 


0.16* 


School Mean Income = $1 5K 


1 IIIMllI 


111! m y. 


0.20 


Eighth Grade Class Size = |||j||| 




m Jill 


t' |a20:i.- 


School Mean Closure = 5 . 


an* 


’ V m 


i|o;i7*' f : : - 


Community Characteristics 


Urban 


0.32* 


0.23 


0.20 


Rural 


0.25* 


0.20 


0.19 


Suburban 


0.21* 


0.21 


0.20 


Northeast 


0.21* 


0.20 


0.19 


North Central 


0.20* 


0.19 


0.19 


West 


0.25* 


0.24* 


0.23* 


South 


0.31* 


0.21* 


0.19* 



Notes: An asterisk (*) indicates probability is significant at the .01 p-level. Unadjusted 
here refers to those probabilities that do not control for any other social context 
measures, while adjusted probabilities control for other factors by assigning other 
variables to the grand mean. Variables not presently available in the NAEP are shaded, 
marked with na and used only to adjust the probabilities in the second column. 
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Population Group Characteristics 

Even after controlling for a wide variety of individual, school, and community 
characteristics, black students have a higher probability of scoring in the bottom 
quartile, but the black-white difference is greatly reduced. The first panel of Table 5.1 
shows that without adjusting for other factors, a black student has about a fifty percent 
chance of scoring in the bottom quartile in mathematics (p = .52). If a black student is 
from an otherwise "average" family, school, and community, however, the probability 
of low achievement is reduced to about one in three. White students have a probability 
of low achievement between .17 and .18, depending on what other social context 
measures are taken into account. Hispanics present a somewhat different picture: 
controlling for wide array of social context variables reduces their probability of being 
in the bottom quartile from quite high (p = .39) to about average (p = .24), although that 
still leaves them somewhat more likely than whites to be low achievers. 

Family Characteristics 

Parental education shows a relationship with the probability of low achievement 
even after controlling for other social context variables, but the relationship is greatly 
reduced from the unadjusted, univariate differences. Controlling for other social 
context variables largely eliminates the higher-than-average probability that children of 
poorly educated parents will score in the bottom quartile, and it somewhat increases the 
probability that the children of college-educated parents will score in that range. 
Nonetheless, even after controlling for a wide variety of variables in the full NELS 
model, the probability that the child of a college-educated parent will score in the 
bottom quartile is 10 percentage points less than the probability for a child of a parent 
with less than a high school education. 

Several family variables are weakly related to low test scores in Table 5.1, 
including single mother households, mother's labor force participation, family income, 
family size, and age of mother at child's birth. Only the first two measures are 
presently in the NAEP. The strong univariate relationship between low income and 
low mathematics scores is substantially diminished when controlling for other 
demographic, family, school, and community characteristics. The unadjusted 
probability that a student who lives in a poor family (e.g., $15,000) is .34, but controlling 



DRAFT 

Identifying Students at Risk 



59 



Berends & Koretz 



for other differences reduces the probability to .21. Similarly, the unadjusted 
probability of students in larger families (p = .42) is reduced in the full NELS model. 

Students who live in families with greater amounts of closure — i.e., parents 
know their child's friends — are less likely to be low achievers, even after controlling for 
other social context variables. Such students have a probability of low achievement of 
.18. 

Language Use, Immigration Status, and Academic Characteristics 

Immigration status, grade retention, and homework are other individual 
characteristics that show independent relationships to the probability of scoring in the 
bottom quartile. Of these three measures, only homework is presently available in 
NAEP. Students who were born outside the U.S. or who had mothers who were 
immigrants are much less likely to score in the bottom quartile, after one controls for 
other social context differences between immigrants and others. Both immigrants and 
children of immigrant mothers have an adjusted probability of .15 of scoring in the 
bottom quartile. These findings sharply contrast the univariate descriptions that did 
not show immigration status differentiating low-achieving students from the 
population. Being held back in school at some point up to the eighth grade is strongly 
related to low test scores. Even after controlling for a variety of other social context 
characteristics, the probability of low mathematics achievement is .42. 

Students who report doing relatively little homework are also more likely to 
score in the bottom quartile, and adjusting for other social context measures reduces but 
does not eliminate this pattern. To illustrate, Figure 5.1 plots the unadjusted and 
adjusted probabilities for low achievement for students who report different levels of 
homework. Controlling for other social context variables lessens the probability that 
students who do very little homework will score in the bottom quartile — bringing that 
probability down to near the population level of .25. Nonetheless, the pattern that 
students doing more homework are less likely to score in the bottom quartile remains 
substantial, albeit weaker than in the case of unadjusted probabilities. Moreover, the 
NAEP model and the full NELS produced nearly identical results. 
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Figure 5.1 



Probability of Low Mathematics Achievement in NELS 
by Hours of Homework Per Week 



1 



Unadjusted 




Adjusted with Variables in NAEP 



Adjusted with Full NELS Model 
X* - 



o 



0 1.5 4 7 10 15 20 

Hours of Homework Per Week 

Other characteristics such as language use and school mobility do not appear to 
have strong independent relationships with low mathematics achievement (see Table 
5.1). While students who report speaking Spanish in their homes have an unadjusted 
probability of low achievement of .44, they have an average probability once other 
factors have been controlled. Moreover, changing schools does not place students at 
risk of low achievement if other social context measures are taken into account. 

School and Community Characteristics 

Students who attend schools with higher mean parent education levels are less 
likely to be low achievers. This is shown in Table 5.1 but is more apparent when a 
wider range of values of parental education and homework are considered (Figures 5.2). 
The unadjusted relationship between parental education and the probability of scoring 
in the bottom quartile is very strong (Figure 5.2). Controlling for a wide variety of 
individual- and school-level social context variables reduces the relationship markedly 
but still leaves s sizable relationship (Figure 5.2). For example, using the NAEP-like 
model, the probability that students will score in the bottom quartile is about .5 in 
schools in which the mean parental educational is only 10 years; the corresponding 
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probability for schools in which the average level of parent education is a college degree 
is .11. In this case, the full NELS model reduces the relationship somewhat more than 
the restricted, NAEP-like model. 



Figure 5.2 

Probability of Low Mathematics Achievement in NELS 
by School Mean Parent Education 




Unadjusted 



Adjusted with Variables in NAEP 

y 

Adjusted with Full NELS Model 



Students in schools that assign greater amounts of homework are also less likely 
to be low achievers (see Figure 5.3). Here again, the very strong unadjusted 
relationship is greatly reduced but remains appreciable after controlling for other 
student and school characteristics. In this instance, the additional variables absent from 
NAEP but included in the full NELS model have no practical impact. These patterns 
stand in contrast to the finding that the amount of homework reported by individual 
students bears little independent relationship to the probability of being a low-achiever. 
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Probability of Low Mathematics Achievement in NELS 
by School Mean Hours of Homework Per Week 




Unadjusted 



Adjusted with Variables in NAEP 

— e- — 

Adjusted with Full NELS Model 



In contrast, the relationship between the racial/ethnic composition of schools 
and the probability of being a low achiever, very striking in terms of unadjusted 
probabilities, essentially evaporates when a wide variety of social context variables are 
taken into account (see Figure 5.4). This is true of both the restricted NAEP-like model 
and the more inclusive full NELS model. It is important to note that among the 
variables held constant in both of these models is the racial/ ethnic identity of individual 
students. 

Other variables that do not have a substantial independent relationship with the 
probability of scoring in the bottom quartile are school sector, the percent of student on 
free or reduced lunch, the size of the eighth grade class in the school, urban locale, and 
region. 
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Probability of Low Mathematics Achievement in NELS 
by School Percentage Black 




Unadjusted 



Adjusted with Variables in NAEP 

Adjusted with Full NELS Model 





Percentage of Black Students in the Student's School 



NET EFFECTS OF MEASURES ADDED TO THE NAEP-LIKE MODEL 

Even though the NAEP-like and full NELS models provided similar adjustments 
to the probabilities of low mathematics scores, an important question remains — what 
would NAEP gain if it added specific measures such as family income? This cannot be 
ascertained solely by comparing the NAEP and NELS models, because the latter adds a 
variety of additional variables, which could obscure the impact of any one of them. 

For the specific purpose of predicting the probability of low achievement, 
adding measures for family income, school income, mother's age at birth of the eighth 
grader, school mobility, family size and grade size — one variable at a time --does not 
substantially improve the prediction of low achievement scores. 

For example, Figure 5.5 shows the probability of low mathematics achievement 
for students with household incomes ranging from $5,000 to $75,000 (in 1987 dollars). 
The unadjusted probabilities show what one would expect: the probability of falling 
into the bottom quartile in mathematics falls steadily as family income rises. In 
contrast, when other social context variables are taken into account (using either the 
NAEP model with income added or the full NELS model), income shows little 
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independent relationship with the probability of being in the bottom quartile. The lines 
for those estimates are nearly flat. 



Figure 5.5 



Probability of Low Mathematics Achievement in NELS 
by Family Income 




» 

Similar patterns emerge for the aggregated measure of school income in Figure 
5.6. Once other variables in NAEP are taken into account the differences in the 
^ probability of low achievement are very small whether the student attends a poor or 

rich school. 



» 
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Figure 5.6 

Probability of Low Mathematics Achievement in NELS 
by School Income 




Average School Income ($1987) 



Mother's age at birth of the eighth grader is only weakly associated with the 
probability of scoring in the bottom quartile even in terms of unadjusted probabilities 
(Figure 5.7). The weakness of this relationship may reflect a poor choice of measures; a 
stronger relationship might have been found had NELS asked the mother's age at time 
of first birth. But when this variable is added to the NAEP model, it has no appreciable 
independent relationship with the probability of scoring in the bottom quartile. 
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Figure 5.7 

Probability of Low Mathematics Achievement in NELS 
by Mother's Age at Birth of Eighth Grader 




Unadjusted 



Adjusted with Variables in NAEP 
— © 

Adjusted with Full NELS Model 

-X : 



Furthermore, measures for family size, eighth grade class size, and school 
mobility, when added one at a time to the NAEP model, do not show appreciable 
independent relationships to low achievement and do not substantially affect the 
prediction of scoring in the bottom quartile. This is illustrated by the weak adjusted 
relationship between number of siblings (e.g., family size) and low achievement scores 
(Figure 5.8). 
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Figure 5.8 

Probability of Low Mathematics Achievement in NELS 
by Number of Siblings 




Unadjusted 



Adjusted with Variables in NAHP 

© 

Adjusted with Full NELS Model 



ASSESSING RELATIONSHIPS AMONG THE SOCIAL CONTEXT MEASURES: 
THE ISSUE OF PROXY MEASURES 

The issue of whether extant measures in NAEP serve as adequate proxies for 
omitted social context variables depends on the purpose of the NAEP. If the main 
question is whether the prediction of low achievement in NAEP is affected by the 
omission of some particular social context measure, then the results in the previous 
section show that the answer is no. The adjusted probabilities in the NAEP-like and full 
NELS models are strikingly similar and adding social context measures that are not 
presently in NAEP reveal only weak relationships with low scores (or no relation at all). 

However, if the question is whether the existing social context measures allow 
researchers and policymakers to identify the same low-achieving students, then 
measures in NAEP must explain a substantial portion of the variable that is omitted. To 
assess the adequacy of the NAEP in this regard, the relationships among the social 
context measures are examined. Table 5.2 shows the results from several ordinary 
regressions in NELS that relate selected family measures to the other demographic and 
family measures and to a wider set of social context measures (e.g., adding school and 
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community characteristics). All of these measures included as independent measures 
are available in NAEP; only the dependent measures are absent from NAEP. 

For example, what is the relationship between the NAEP-like social context 
measures and family income? The first two rows of Table 5.2 show that other social 
context measures explain between about one-quarter to one-third of the variance in 
income. When family income is regressed on other demographic and family measures 
(e.g., parent education, single parent household, and mother's labor force participation), 
the R 2 is .25, and when further measures for school and community are added to the 
regression, the R 2 increased to .31. Because our analyses in the previous section showed 
that income has a strong unadjusted association with low test scores and a weak one 
once other social context measures are taken into account, that part of family income 
that is important for predicting low scores is about one-third. Therefore, if a purpose of 
the NAEP is to track the test scores of poor students or poor schools, adding family 
income data is essential. 



Table 5.2 

Relationships Between Specific Family and 
Sets of Social Context Measures Presently in NAEP 



Ordinary Regression Model Adjusted R 2 

Income on demographic & other family characteristics .25 

Income on all other social context measures .31 

Family size on demographic & family characteristics .07 

Family size on all other social context measures .07 

Mother's age at child's birth on demographic & family characteristics .06 

Mother's age at child's birth on all other social context measures .08 

Family closure on demographic & family characteristics .12 

Family closure on all other social context measures .14 



The social context measures currently in NAEP do an even worse job of 
explaining the variance in family measures such as family size, mother's age at birth of 
the child, and family closure. At most, a little more than one-tenth of the variance is 
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explained by the other social context measures available in NAEP (i.e., the R 2 for family 
closure is .14). 

The relationships among the school measures are much higher, but even so, the 
extant NAEP measures do not provide good proxy measures for school poverty or 
school closure. The associations between existing social context measures and global 
characteristics of schools — including indicators of socioeconomic status, minority 
composition, and climate — are shown in Table 5.3. Two-thirds of the variance in school 
mean income is explained by other demographic, family, school and community 
measures (R 2 = .67). The extant measures of social context explain about one-half of the 
variance in school closure (R 2 = .53). Although there is a significant degree of 
collinearity between the school measures in NAEP and those absent from it, a 
substantial portion of the variance is left unexplained by the NAEP measures. 

Therefore, , the current NAEP measures are not sufficient proxies for those omitted if 
the purpose is to identify the same groups of students. 

Table 5.3 also reveals further collinear relationships among the school measures 
currently available in NAEP. Between fifty to sixty percent of the variance in the school 
socioeconomic composition measures (e.g., parent education and percentage of students 
in school are free or reduced lunch) is explained by existing NAEP social context 
characteristics. The variance explained in the school minority composition measures is 
much higher if demographic, family, school, and community measures are included. 
School mean homework is not explained well by the other social context measures in 
NAEP (R 2 = .20). 
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Table 5.3 



Relationships Between Specific School and 
Sets of Social Context Measures Presently in NAEP 



Ordinary Regression Model Adjusted R 2 



School Socioeconomic Status 

School mean income on other school characteristics .63 

School mean income on all other social context measures .67 

School closure on other school characteristics .43 

School closure on all other social context measures .53 

School mean parent education on other school characteristics .50 

School mean parent education on all other social context measures .59 

% Free lunch on other school characteristics .51 

% Free lunch on all other social context measures .54 

School Minority Composition 

% black on other school characteristics .26 

% black on all other social context measures .59 

% Hispanic on other school characteristics .25 

% Hispanic on all other social context measures .58 

School Climate 

School mean homework on other school characteristics .17 

School mean homework on all other social context measures .20 
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6 . 



CONCLUSIONS 



The primary objective of this study was to assess the adequacy of NAEP's 
family, school and community measures for describing students at risk of educational 
failure (i.e., scoring in the bottom quartile or decile of the achievement distribution). 
NAEP's adequacy, however, depends on the purposes for which the social context 
measures are used. This study focused on two purposes: 

• describing low-achieving students according to this group's family, school, 
and community characteristics and 

• predicting low achievement scores based on students' family, school, and 
community characteristics. 

Our analysis of the latter purpose differs from the majority of existing research that 
examines the independent effects of social context on mean achievement levels rather 
than low achievement levels (e.g., bottom quartile or decile). Most of our findings and 
implications for the NAEP have been discussed in the summary sections of the previous 
chapters and the summary of this report. However, because of the extent of the various 
analyses, a few general concluding comments are in order. 

For both purposes several measures in NAEP are useful for describing at-risk 
students. For example, population group characteristics, parents' education 
attainments, school-level parent education, and school mean homework are measures 
are important for differentiating low-achieving groups from the population as a whole 
and for independently predicting low achievement scores. 

However, some measures in NAEP that differentiate low-achieving groups do 
not predict low achievement very well once other factors have been controlled. For 
example, measures for single parent households, school minority composition, and 
percentage in the school on free lunch differentiated low-achieving groups from the 
population, but they were not independently associated with low achievement scores. 

For differentiating low-achieving students from the population, NAEP should 
add measures for family income, family size, mother's age at birth of first child, family 
closure, student mobility between schools, and school income and closure. Even 
though these measures do not appreciably add to NAEP's ability to predict low test 
scores, they are important for providing univariate descriptions of students at risk of 
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educational failure. Although absent from NAEP, grade retention is important for both 
predicting low achievement and differentiating low achievers as a group. 

Our results raise the question whether the existing social context measures in 
NAEP are adequate proxies for omitted variables such as family income. Here too, the 
answer depends on the purpose to which the measures are put. If the goal is only to 
predict the probability that a student will be a low achiever, our analyses suggest that a 
large enough number of the NAEP social context variables, taken together, are a fairly 
good proxy for income. That is, adding an income variable to NAEP will not 
dramatically change its ability to predict the probability of scoring in the bottom 
quartile. However, further analyses show that for other purposes — for example, to 
monitor the achievement of poor students — the variables included in the NAEP do not 
provide an adequate proxy for income. For instance, because family income has a 
strong unadjusted relationship with low test scores and a weak relationship once other 
factors are controlled, our analyses show that the portion of family income that matters 
in predicting the probability of low achievement is one-third of the variance in income 
that is related to the other social context variables. 

The extent of the relationships among the school measures is much greater them 
the relationships between, for example, family income and other social context 
characteristics. Even for school measures, however, existing NAEP measures serve as 
poor proxies if the intent is to describe students according to school poverty (based on 
aggregated family income) or school closure (e.g., "social capital," or connections 
among families in the school). At most, two-thirds of the variance of these measures is 
explained by other demographic, family, school, and community characteristics that are 
available in NAEP. With such a substantial portion of the variance is left unexplained, 
the current NAEP can only provide inadequate proxies for these omitted school 
measures. Therefore, if the aim of NAEP is to allow researchers and policymakers to 
identify the same low achieving students, then the NAEP currently contains inadequate 
proxy measures for both family and school contexts. 
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Table A .1 

Percentage of Achievement Groups with Certain Social Context Characteristics in NAEP. 
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Percentage of Achievement Groups with Certain Social Context Characteristics in NAEP. 
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Percentage of Achievement Groups with Certain Social Context Characteristics in NELS. 
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Percentage of Achievement Groups with Certain Social Context Characteristics in NELS. 
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Table B.l 

Means and Standard Deviations for Social Context Measures in Logistic Regression Models 



Variable Label 


Mean 


SD 


Black 


0.127 


0.331 


Hispanic 


0.094 


0.290 


Other 


0.044 


0.205 


Female 


0.501 


0.497 


Mother's Education Less High School 


0.148 


0.353 


Mother's Education Some College 


0.404 


0.487 


Mother's Education College Graduate 


0.160 


0.364 


Father's Education Less High School 


0.163 


0.366 


Father's Education Some College 


0.335 


0.469 


Father's Education College Graduate 


0.238 


0.423 


Mother is Single Parent 


0.162 


0.359 


Mother Works Part Time 


0.510 


0.489 


Mother Works Full Time 


0.176 


0.372 


Family Income (1987 $00,000s ) 


3.870 


3.504 


Number of Siblings 


2.264 


1.537 


Mother's Age at Birth of 8th Grader 


25.590 


5.865 


Family Closure 


3.625 


1.764 


Spanish Usually Spoken at Home 


0.054 


0.224 


Language Other than English Spoken at Home 


0.032 


0.174 


Student was Bom Outside U.S. 


0.047 


0.209 


Mother was Bom Outside U.S. 


0.104 


0.299 


Change Schools More than Once 


0.336 


0.469 


Ever Held Back 


0.192 


0.388 


Homework Hours /Week 


5.724 


4.535 


Catholic School 


0.074 


0.259 


Other Private School 


0.043 


0.203 


Percent on Free Lunch 


24.011 


22.909 


Percent Black 


13.534 


23.291 


Percent Hispanic 


8.407 


19.341 


School Parent Education 


13.710 


0.929 


School Homework 


5.715 


1.609 


School Income 


3.863 


2.031 


8th Grade Class Size 


204.974 


127.685 


School Closure 


3.579 


0.747 


Urban 


0.239 


0.423 


Rural 


0.326 


0.466 


Northeast 


0.185 


0.386 


North Central 


0.269 


0.440 


West 


0.181 


0.383 


Mother's Education Missing 


0.007 


0.082 


Father's Education Missing 


0.059. ' 


0.235 


Family Structure Missing 


0.038 ~ 


0.189 


Mother's Labor Force Participation Missing 


0.030 


0.169 


Family Income Missing 


0.043 


0.202 


Family Size Missing 


0.016 


0.123 


Language Use Missing 


0.007 


0.085 


Student's Immigrant Status Missing 


0.016 


0.126 


Mother's Immigrant Status Missing 


0.024 


0.153 


School Mobility Missing 


0.018 


0.132 


School Retention Missing 


0.018 


0.132 


Homework Missing 


0.062 


0.239 




Table B.2 

Social Context Predictors of Bottom Mathematics Quartile in Full NELS Model. 



Variable 


Parameter 

Estimate 


Standard 

Error 


Wald 

£ 


Pr > 1 } 


Odds 

Ratio 


Intercept 


3.6745 


0.473 


60.3571 


0.0001 


39.431 


Black 


0.9389 


0.0665 


199.5277 


0.0001 


2.557 


Hispanic 


0.3996 


0.082 


23.7473 


0.0001 


1.491 


Other 


0.5122 


0.1003 


26.0847 


0.0001 


1.669 


Female 


0.1835 


0.0372 


24.3883 


0.0001 


1.201 


Mother's Education Less High School 


0.1408 


0.0581 


5.8621 


0.0155 


1.151 


Mother's Education Some College 


-0.1193 


0.046 


6.723 


0.0095 


0.888 


Mother's Education College Graduate 


-0.5246 


0.081 


41.9095 


0.0001 


0.592 


Father's Education Less High School 


0.1363 


0.056 


5.9262 


0.0149 


1.146 


Father's Education Some College 


-0.202 


0.0485 


17.3495 


0.0001 


0.817 


Father's Education College Graduate 


-0.4759 


0.0692 


47.2716 


0.0001 


0.621 


Mother is Single Parent 


-0.104 


0.0562 


3.422 


0.0643 


0.901 


Mother Works Part Time 


-0.0548 


0.0424 


1.6701 


0.1962 


0.947 


Mother Works Full Time 


-0.0766 


0.0588 


1.6977 


0.1926 


0.926 


Family Income (1987 $00,000s ) 


-0.0368 


0.00896 


16.887 


0.0001 


0.964 


Number of Siblings 


-0.00641 


0.0119 


0.2891 


0.5908 


0.994 


Mother's Age at Birth of 8th Grader 


0.000788 


0.00314 


0.0629 


0.8019 


1.001 


Family Closure 


-0.0717 


0.0109 


43.6257 


0.0001 


0.931 


Spanish Usually Spoken at Home 


0.1933 


0.1 


3.7365 


0.0532 


1.213 


Language Other than English Spoken at Home 


0.1812 


0.1169 


2.401 


0.1213 


1.199 


Student was Bom Outside U.S. 


-0.3252 


0.1036 


9.8579 


0.0017 


0.722 


Mother was Bom Outside U.S. 


-0.348 


0.0845 


16.9592 


0.0001 


0.706 


Change Schools More than Once 


0.0336 


0.0405 


0.6868 


0.4073 


1.034 


Ever Held Back 


1.3117 


0.0432 


922.7564 


0.0001 


3.712 


Homework Hours /Week 


-0.0498 


0.00473 


111.0167 


0.0001 


0.951 


Catholic School 


0.2939 


0.0934 


9.9021 


0.0017 


1.342 


Other Private School 


0.0204 


0.1431 


0.0204 


0.8865 


1.021 


Percent on Free Lunch 


0.000653 


0.00108 


0.364 


0.5463 


1.001 


Percent Black 


0.00357 


0.00111 


10.2389 


0.0014 


1.004 


Percent Hispanic 


-0.00197 


0.00135 


2.1403 


0.1435 


0.998 


School Parent Education 


-0.2842 


0.0378 


565396 


0.0001 


0.753 


School Homework 


-0.0567 


0.0139 


16.6297 


0.0001 


0.945 


School Income 


-0.0166 


0.0211 


0.621 


0.4307 


0.983 


8th Grade Class Size 


-0.00016 


0.000191 


0.7085 


0.3999 


1.000 


School Closure 


-0.1075 


0.0364 


8.7057 


0.0032 


0.898 


Urban 


-0.00196 


0.0525 


0.0014 


0.9702 


0.998 


Rural 


-0.0513 


0.0485 


1.118 


0.2904 


0.95 


Northeast 


-0.0146 


0.0565 


0.0667 


0.7962 


0.986 


North Central 


-0.056 


0.0505 


1.2305 


0.2673 


0.946 


West 


02337 


0.0586 


15.9.038 


0.0001 


1.263 


Mother's Education Missing 


0.3715 


02194 


2.8667 


0.0904 


1.45 


Father's Education Missing 


0.2077 


0.0773 


7.2253 


0.0072 


1.231 


Family Structure Missing 


0.189 


0.0876 


4.6535 


0.031 


1.208 


Mother's Labor Force Participation Missing 


0.1462 


0.1095 


1.783 


0.1818 


1.157 


Family Income Missing 


-0.0878 


0.0907 


0.9376 


03329 


0.916 


Family Size Missing 


0.0501 


0.172 


0.0849 


0.7707 


1.051 


Language Use Missing 


0.4007 


0.1965 


4.1571 


0.0415 


1.493 


Student's Immigrant Status Missing 


0.1932 


0.1885 


1.0506 


0.3054 


1.213 


Mother's Immigrant Status Missing 


0.0234 


0.1437 


0.0266 


0.8705 


1.024 


School Mobility Missing 


02777 


02204 


1.5872 


0.2077 


1.32 


School Retention Missing 


0.1387 


0.2252 


0.3793 


0.538 


1.149 


Homework Missing 


02413 


0.0686 


24.7244 


0.0001 


1.407 



Table B.3 

Social Context Predictors of Bottom Mathematics Quartile in NAEP-Like Model 



Variable 


Parameter 

Estimate 


Standard 

Error 


Wald 

X 2 


Pr>x 2 


Odds 

Ratio 


Intercept 


42132 


0.4013 


110.2047 


0.0001 


67.573 


Black 


0.9922 


0.0632 


246.6833 


0.0001 


2.697 


Hispanic 


03823 


0.0789 


23.4781 


0.0001 


1.466 


Other 


0.3705 


0.0928 


15.9414 


0.0001 


1.448 


Female 


0.0103 


0.0352 


0.0856 


0.7699 


1.01 


Mother's Education Less High School 


02782 


0.0554 


25.2263 


0.0001 


1.321 


Mother's Education Some College 


-0.1407 


0.0442 


10.1191 


0.0015 


0.869 


Mother's Education College Graduate 


-0.6564 


0.0781 


70.5748 


0.0001 


0.519 


Father's Education Less High School 


0.2293 


0.0536 


18.3267 


0.0001 


1.258 


Father's Education Some College 


-02231 


0.0466 


22.9069 


0.0001 


0.8 


Father's Education College Graduate 


-0.647 


0.0658 


96.5734 


0.0001 


0.524 


Mother is Single Parent 


0.0344 


0.0527 


0.4268 


0.5136 


1.035 


Mother Works Part Time 


-0.1026 


0.0405 


6.4185 


0.0113 


0.902 


Mother Works Full Time 


-0.146 


0.0565 


6.6768 


0.0098 


0.864 


Spanish Usually Spoken at Home 


0.0464 


0.0935 


0.2463 


0.6197 


1.047 


Language Other than English Spoken at Home 


0.0577 


0.1089 


0.2807 


0.5963 


1.059 


Homework Hours /Week 


-0.0514 


0.00458 


125.9376 


0.0001 


0.95 


Catholic School 


0.0736 


0.0789 


0.8697 


0.351 


1.076 


Other Private School 


-0.1504 


0.1293 


1.3524 


0.2449 


0.86 


Percent on Free Lunch 


0.00135 


0.00102 


1.7381 


0.1874 


1.001 


Percent Black 


0.00344 


0.00104 


10.8733 


0.001 


1.003 


Percent Hispanic 


-0.00383 


0.00127 


9.0328 


0.0027 


0.996 


School Parent Education 


-0.3536 


0.0293 


146.0277 


0.0001 


0.702 


School Homework 


-0.0512 


0.0131 


15.2531 


0.0001 


0.95 


Urban 


0.091 


0.0489 


3.4653 


0.0627 


1.095 


Rural 


-0.0874 


0.0437 


3.9924 


0.0457 


0.916 


Northeast 


-0.0959 


0.0533 


32379 


0.072 


0.909 


North Central 


-0.1174 


0.0475 


6.1079 


0.0135 


0.889 


West 


0.1459 


0.0555 


6.9211 


0.0085 


1.157 


Mother's Education Missing 


0.255 


0.2107 


1.4648 


0.2262 


1.29 


Father's Education Missing 


02495 


0.0745 


11.2045 


0.0008 


1.283 


Family Structure Missing 


0.2901 


0.0838 


11.98 


0.0005 


1.337 


Mother's Labor Force Participation Missing 


0.3318 


0.104 


10.1816 


0.0014 


1.393 


Language Use Missing 


0.4929 


0.1852 


7.0798 


0.0078 


1.637 


Homework Missing 


0.4977 


0.0653 


58.1105 


0.0001 


1.645 



Appendix B.4 

Individual-Level Correlations Among Selected Family Characteristics in NELS (N=20,055) 



§ 



Family Measure 


(1) 


_J2}_ 


_J3)_ 


_J4L_ 


(5) 


(6) 


(1) Income 


1.00 












(2) Mother's Education 


.34 


1.00 










(3) Father's Education 


.40 


.59 


1.00 








(4) Family Size 


-.19 


-.17 


-.17 


1.00 






(5) Mother's Age at 8th Grader's Birth 


.12 


.09 


.09 


.11 


1.00 




(6) Family Closure 


.19 


.23 


.21 


-.11 


.08 


1.00 
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